-- Logs begin at Thu 2022-05-19 13:47:57 UTC, end at Thu 2022-05-19 15:17:25 UTC. -- May 19 14:19:49.768352 dv-cinder-842542-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:19:49.778336 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:19:49 2022] *** May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: nodename: dv-cinder-842542-1-iscsi May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: machine: x86_64 May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: clock source: unix May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: pcre jit disabled May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: detected number of CPU cores: 4 May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: current working directory: / May 19 14:19:49.780586 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: detected binary path: /usr/bin/uwsgi-core May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: your processes number limit is 47680 May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: your memory page size is 4096 bytes May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: detected max file descriptor number: 2048 May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: lock engine: pthread robust mutexes May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: thunder lock: enabled May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:19:49.781127 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:19:49.838001 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: Python main interpreter initialized at 0x559afb2fa9c0 May 19 14:19:49.838001 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: python threads support enabled May 19 14:19:49.838001 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: your server socket listen backlog is limited to 100 connections May 19 14:19:49.838001 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: your mercy for graceful operations on workers is 90 seconds May 19 14:19:49.838324 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:19:49.838399 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: *** Operational MODE: preforking *** May 19 14:19:49.838695 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: *** uWSGI is running in multiple interpreter mode *** May 19 14:19:49.838695 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: spawned uWSGI master process (pid: 80288) May 19 14:19:49.838890 dv-cinder-842542-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:19:49.839461 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: spawned uWSGI worker 1 (pid: 80298, cores: 1) May 19 14:19:49.839809 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: spawned uWSGI worker 2 (pid: 80299, cores: 1) May 19 14:19:49.844805 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:19:55.377833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:19:55.377833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: from cryptography.utils import int_from_bytes May 19 14:19:55.378656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:19:55.378656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: from cryptography.utils import int_from_bytes May 19 14:19:55.421679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:19:55.421679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes May 19 14:19:55.422242 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:19:55.422242 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes May 19 14:19:55.428666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:19:55.428976 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:19:55.429305 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] command line args: [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:19:55.429812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:19:55.430076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ================================================================================ {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:19:55.430519 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] admin_token = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.430931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] config_dir = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.431172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.431378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] config_source = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.431550 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] debug = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.431801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.432164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.432342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.432728 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.433173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.433612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.433799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_config_append = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.433989 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.434182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_dir = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.434341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_file = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.434600 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.434852 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.435100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.435346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.435510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.435694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.435926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.436170 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.436403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.436636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.436866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.437096 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.437366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.437609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.437899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.438383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.438584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] publish_errors = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.438806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.439020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.439196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.439441 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.439703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.439952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] standard_threads = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.440161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.440403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.440576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.440747 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] use_journal = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.440919 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] use_json = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.441090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] use_stderr = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.441277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] use_syslog = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.441508 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.441763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.442017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.442254 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.442490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.442736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.442974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.443219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.443452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.443698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.443941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.444188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.444419 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.444663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.444896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.445142 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.445414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.445650 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.445905 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.446158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.446404 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.446641 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.446893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.447153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.447407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.447655 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.447886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.448117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.448346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.448582 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.448829 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.449075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.449324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.449554 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.449799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.450049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.450285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.450541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.450771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.451008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.451240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.451468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.451698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.451927 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.452159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.452392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.452640 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.452870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.453108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.453357 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.453593 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.453826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.454072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.454301 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.454537 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.454772 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.455005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.455292 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.455530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.455758 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.456005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.456238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.456475 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.456702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.456960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.457193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.457446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.457681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.457913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.458177 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.458409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.458657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.458897 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.459128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.459359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.459589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.459831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.460065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.460296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.460539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.460770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.461004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.461260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.461486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.461718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.461974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.462208 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.462444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.463739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.463979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.464215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.464450 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.464684 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.464910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.465136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.465389 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.465622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.465850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.466087 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.466319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.466565 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.466799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.467174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.467435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.467673 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.467901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.external = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.468128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.468421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.468651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.468879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.password = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.469106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] auth.token = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.469367 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.469603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.469848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.470076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.470322 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.470567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.470814 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.471051 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.471298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.471530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:19:55.471649 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.471763 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:19:55.471882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.472003 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] command line args: [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:19:55.472115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.472227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:19:55.472337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.472455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ================================================================================ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:19:55.472579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.472692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.472803 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.472920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_dir = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.472920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.473139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.473276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_source = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.473391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.473502 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] debug = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.473613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.473723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.473723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.473988 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.474117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.474236 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.474348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.474471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.474585 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.474682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.474776 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.474869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.474962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_config_append = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.474962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.475153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.475248 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_dir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.475350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.475446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.475571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.475683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.475785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.475870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.475979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.476091 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.476203 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.476324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.476324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.476543 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.476661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.476775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.476895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.477008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.477120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.477251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.477371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.477494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.477608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.477727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.477849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.477953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.478064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.478175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.478286 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.478397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.478523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.478642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.478755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.478865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.478865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.479084 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.479198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.479319 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] publish_errors = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.479431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.479543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.479663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.479784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.479784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.479976 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.480087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.480207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.480312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.480429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.480533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] standard_threads = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.480643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.480755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.480874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.480987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.481099 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.481219 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.481364 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.481488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_journal = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.481488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_json = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.481710 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.481822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_stderr = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.481822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_syslog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.482031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.482143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:19:55.482336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.482454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.482572 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.482685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.482796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.482907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.483925 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484036 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.484939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.485883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.486894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487006 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.487925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488157 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488298 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488777 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.488890 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489650 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489779 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.489904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490131 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490244 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490469 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490469 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.490895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491695 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491809 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.491931 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492164 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.492980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493098 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493595 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493708 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.493943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.494887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.495928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496041 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496473 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496706 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.496951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497064 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497178 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497358 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.497939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498636 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.498975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499097 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499325 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499438 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.499900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500341 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500803 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.500917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] role.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501650 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] role.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.501886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.502987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503213 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503679 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.503905 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504027 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504142 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504255 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.504950 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505207 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505207 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505737 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.505962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506655 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506768 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.506888 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507009 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.external = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507583 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.507924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.password = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.token = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508499 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508838 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.508959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509685 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509799 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.509912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510375 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510828 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.510949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511064 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511177 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511290 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511516 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511752 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.511980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512440 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512559 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512673 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.512899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.513887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514001 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514119 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.connection = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514242 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514365 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514480 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514593 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514828 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.514942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515056 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515169 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.515857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516047 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.516971 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.517944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518057 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.518748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.519684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.519684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.519684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.519684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.519684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.519684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.520297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.520297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.520297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.520611 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.520733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.520882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.521132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.521383 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.521624 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.521865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.522087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.522318 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.522539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.522771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.523012 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.523246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.523474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.523948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.524188 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.524414 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.524637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.524878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.525118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.525370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.525601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.525829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.526052 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.526285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.526537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.526763 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.527008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.527240 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.527478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.527703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.527924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.528150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.528375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.528602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.528821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.529446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.529686 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.529923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.530159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.530396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.530622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.530859 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.531104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.531316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.531544 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.531771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.532008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.532247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.532475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.532719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.532940 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.533174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.533506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.533738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.533969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.534208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.534443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.534668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.534896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.535127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.535368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.535616 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.535839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.536081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.536323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.536551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.536790 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.537019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.537269 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.537495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.537735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.537957 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.538198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.538440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.538682 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.538922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.539161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.539399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.539637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.539891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.540122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.540362 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.540607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.540847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.541085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.541525 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.541756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.541997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.542238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.542478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.542717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.542942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.543188 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.543434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.543668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.543891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.544114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.544337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.544579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.544827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.545067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.545329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.545558 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.545782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.546011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.546238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.546478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.546707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.546955 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.547185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.547308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.547428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.547552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.547669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.547786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.547900 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.548098 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.548327 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.548557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.548784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.549048 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.549317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.549543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.549779 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.550064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.550317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.550554 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.550801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.551031 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.551263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.551492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.551721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.551950 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.552179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.552416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.552645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.552884 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.553112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.553388 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.553618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.553856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.554077 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.554309 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.554537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:19:55.554700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:19:55.556254 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.579504 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.579725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.579923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.584327 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.584561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.589829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.608008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.608212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.636748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.644676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.687248 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.687444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.691676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.691903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.751529 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.751682 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.751763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.751829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.751915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752313 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.752742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.775544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.775682 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.775782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.775893 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776376 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776532 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776596 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776790 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.776942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.788371 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.788500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.801192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.801407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.801647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.801864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.802134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.802282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.802448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.802922 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.803154 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.803369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.803567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.803771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.803993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.804184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:19:55.819656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.819785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.819864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:19:55.819940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:19:55.820168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.820296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.820373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:19:55.820444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:19:55.820634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.820767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.820860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.820941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.821102 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.821230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.821336 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.821433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.832452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.832723 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.832986 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.833291 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.833932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.834196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.834458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.834738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.841405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.842835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.844680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.844954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.845231 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.845633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.845964 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.846223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.846348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.846463 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.846554 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.846760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.846887 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.846982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.847179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.847306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.847385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.847538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.847678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.847898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.848159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.848414 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.848665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.848923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.874061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.875732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.876038 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.877528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.879182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.879449 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.881711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.881986 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.882266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.886710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.887049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.887261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.894457 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.894736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.896038 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.896456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.899564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.899845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.900325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.900655 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.900936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.901210 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.901488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.901571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.901754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.901833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.902030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.902301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.902566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.902839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.903114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.903571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.921554 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.921834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.925479 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.925760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.926456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.930087 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.930256 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.930353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.933904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.936094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.937003 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.939140 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.939267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.942808 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.942956 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.943273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.943549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.943842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.944167 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.944475 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.946516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.946805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.947080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.947348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.947612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.947881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.963604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.963967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.964232 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.964490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.967316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.967686 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.967948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.968214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.972426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.974172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.974447 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.976139 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.977491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.977764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.977908 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.978028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.978119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.978324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.978606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.981114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.981403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.981682 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.981968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.982235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.992348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.992636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.992911 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.993185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.993480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.993740 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.994012 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.996568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:55.996844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.997125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.997414 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.997686 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.997948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:55.998214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.012120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.015406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.016204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.018810 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.019569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.020406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.022075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.022361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.022634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.022886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.024472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.026123 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.026398 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.026674 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.030423 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.032052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.034471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.035322 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.035608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.035885 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.036037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.036133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.039300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.039589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.039875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.040150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.045418 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.045744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.046056 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.049398 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.049683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.049948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.054043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.054314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.054601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.054874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.055148 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.055418 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.055687 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.055936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.056209 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.056469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.056720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.056978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.057267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.057532 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.057960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:19:56.058231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.058485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.058755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.059020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.059281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.059552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.059820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.060092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.060350 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.060613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.060871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.061123 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.061413 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:19:56.096598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559afb2fa9c0 pid: 80299 (default app) May 19 14:19:56.100103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191540aa-63ac-48b5-ba16-d5fce80d38a8 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:19:56.100229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559afb2fa9c0 pid: 80298 (default app) May 19 14:19:56.100402 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191540aa-63ac-48b5-ba16-d5fce80d38a8 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:19:56.100677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191540aa-63ac-48b5-ba16-d5fce80d38a8 None None] PATH_INFO: `/v3` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:19:56.104922 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Thu May 19 14:19:56 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:04.134493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:04.135092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:04.135092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:04.139493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:04.146012 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:04.146781 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:04.146925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:04.219127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: INFO dbcounter [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] Registered counter for database keystone May 19 14:20:04.244563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG oslo_db.sqlalchemy.engines [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80299) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:20:04.256865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writer thread running {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:20:04.440881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG passlib.handlers.bcrypt [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80299) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:20:04.441315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG passlib.handlers.bcrypt [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80299) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:20:04.497385 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:04.498942 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:04.629596 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:20:04.905657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:04.981634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: INFO dbcounter [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] Registered counter for database keystone May 19 14:20:05.003814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG oslo_db.sqlalchemy.engines [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80298) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:20:05.013594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writer thread running {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:20:05.134849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56e6a0-14e4-402e-8b00-5ae992a5fb64 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:05.143676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b58c8de9649042479ca5651859a02fd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:05.144359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:05.144640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:05.144909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:05.154628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:05.205344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:20:05.205637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bf511-629e-401d-ac2f-e946c1424814 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:05.216047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 14:20:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:20:05.223156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:05.243829 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538a3f6-24df-40d1-baee-76d46e42baf6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:05.249618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b58c8de9649042479ca5651859a02fd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:05.250755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:05.251033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:05.251313 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] PATH_INFO: `/v3/projects/b58c8de9649042479ca5651859a02fd6` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:05.253040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] RBAC: Authorizing `identity:get_project(project_id=b58c8de9649042479ca5651859a02fd6)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:05.416957 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:20:05.417319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a7888-9b97-4cf0-9ce2-22784b96c8fc admin admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:05.422077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Thu May 19 14:20:05 2022] GET /identity/v3/projects/b58c8de9649042479ca5651859a02fd6 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:07.612557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb637-cc3b-4fc6-a086-f1d1387354ff None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:07.613140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb637-cc3b-4fc6-a086-f1d1387354ff None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:07.613140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb637-cc3b-4fc6-a086-f1d1387354ff None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:07.617383 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:07.624028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:07.624699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:07.624905 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:07.647782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:07.649353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:07.659636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:20:07.693604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbb637-cc3b-4fc6-a086-f1d1387354ff None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:07.707899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbb637-cc3b-4fc6-a086-f1d1387354ff None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:07.713366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77624-260f-494a-9b3c-db1b2b7eb915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b58c8de9649042479ca5651859a02fd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:07.714002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77624-260f-494a-9b3c-db1b2b7eb915 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:07.714288 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77624-260f-494a-9b3c-db1b2b7eb915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:07.714563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77624-260f-494a-9b3c-db1b2b7eb915 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:07.723040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77624-260f-494a-9b3c-db1b2b7eb915 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:07.724826 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77624-260f-494a-9b3c-db1b2b7eb915 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:07.733869 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 14:20:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:20:07.739805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:07.753843 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4b59-f496-4a01-a08b-bb1c77c09ec7 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:07.760158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988438f-e3ae-4749-9c5c-f729e474db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b58c8de9649042479ca5651859a02fd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:07.761035 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988438f-e3ae-4749-9c5c-f729e474db51 admin admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:07.761346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988438f-e3ae-4749-9c5c-f729e474db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:07.761622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988438f-e3ae-4749-9c5c-f729e474db51 admin admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:07.763178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988438f-e3ae-4749-9c5c-f729e474db51 admin admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:07.765021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988438f-e3ae-4749-9c5c-f729e474db51 admin admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:07.766625 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Thu May 19 14:20:07 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:10.038580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eaa817-9251-4ba2-8cdb-0ee9957c3c41 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.039149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eaa817-9251-4ba2-8cdb-0ee9957c3c41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.039149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eaa817-9251-4ba2-8cdb-0ee9957c3c41 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.040484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:10.041064 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5fce-4801-4cce-a961-221654b6eb6c None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.041542 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5fce-4801-4cce-a961-221654b6eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.041640 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5fce-4801-4cce-a961-221654b6eb6c None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.042993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:10.045559 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.045846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.046121 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.048727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.049120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.049617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.071846 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:10.073034 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:10.092236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80298) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:20:10.092663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80298) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:20:10.138669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:10.140241 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:10.155079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 7/12] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:10.176974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.209340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:10.249678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 6/13] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:10.272582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.293256 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6463a-3eeb-4308-bd9b-d418ae0c24da None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.326817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.327475 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.327757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.328027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.379815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe239cdc-134b-4bdc-9c20-9124e456e96c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.382391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:10.397428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:10.398918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:10.417352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.418653 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.418938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.419210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.421738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:10.431905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:10.443175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:10.444096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:10.445482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:10.450684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.499333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 8/15] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:20:10.505063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.548071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80e6a-6421-44e6-b8d1-97e8d7a36351 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.583328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.583975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.584257 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.584529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.585317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:10.585626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:10.587076 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:10.599190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 8/16] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:10.608199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.612780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e86975-a06d-41c0-8c77-f8ff23177c78 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.636203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:10.636624 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:10.648988 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.649844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.650125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.650398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.661072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:10.662594 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:20:10.682836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:10.683690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:10.684739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:20:10.684739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:20:10.684739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:20:10.684739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:10.684739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 9/17] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:10.690048 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.708920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc02735-4b3d-4cfd-b1a4-a579c09f2142 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.742542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.743430 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.743712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.743985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.745590 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:10.747084 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:10.748720 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 9/18] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:10.754122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.790088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aab6a8-ec4d-4d1b-90aa-444371fa7d32 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.824516 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.825186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.825496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.825772 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.826530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:10.826835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:10.828245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:10.844759 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:10.845002 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 10/19] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:10.852382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a9915-1c0c-45d6-8b66-f21322849221 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:10.889231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafaf7c-11b7-411b-b4dd-304880d0ead6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:10.890095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafaf7c-11b7-411b-b4dd-304880d0ead6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:10.890370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafaf7c-11b7-411b-b4dd-304880d0ead6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:10.890644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafaf7c-11b7-411b-b4dd-304880d0ead6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:10.891894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafaf7c-11b7-411b-b4dd-304880d0ead6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:10.893450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafaf7c-11b7-411b-b4dd-304880d0ead6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:10.895106 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 10/20] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:10.901348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:10.941282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:11.017725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ba46a-2fd3-4d08-8521-01d18c98b446 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:11.054048 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:11.054697 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:11.054981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:11.055255 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:11.056083 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.domain.id=default, user.id=575363a61b9b436f9c41ee0b0af8ab40) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:11.056396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:11.057912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f88d-c62c-443f-9f89-1e4ce8600612 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:11.064999 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 11/21] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 14:20:10 2022] GET /identity/v3/role_assignments?user.id=575363a61b9b436f9c41ee0b0af8ab40&scope.domain.id=default&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:13.355964 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641773b-215b-4fdf-bf49-095f07b94cd2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:13.356988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641773b-215b-4fdf-bf49-095f07b94cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:13.357506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641773b-215b-4fdf-bf49-095f07b94cd2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:13.359101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 11/22] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:13.364561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:13.365456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:13.365950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:13.387690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:13.389350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:13.471623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 12/23] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:13.506247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641773b-215b-4fdf-bf49-095f07b94cd2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:13.639165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641773b-215b-4fdf-bf49-095f07b94cd2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:13.685936 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c026e-0797-4ae4-a014-2135b148ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:13.687104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c026e-0797-4ae4-a014-2135b148ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:13.687578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c026e-0797-4ae4-a014-2135b148ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:13.688041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c026e-0797-4ae4-a014-2135b148ef31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:13.703840 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c026e-0797-4ae4-a014-2135b148ef31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c41c026e-0797-4ae4-a014-2135b148ef31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:13.711810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:13.712667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:13.713481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:13.715831 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 12/24] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:13.721770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:13.749749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:13.758547 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e690f-0d8f-4959-a63a-47fbc13e4821 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:13.758842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e690f-0d8f-4959-a63a-47fbc13e4821 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:13.759325 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e690f-0d8f-4959-a63a-47fbc13e4821 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:13.763657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 13/25] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:13.770965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728debc-4ea2-4fb5-a9fa-5e0dcc30882a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:13.771253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728debc-4ea2-4fb5-a9fa-5e0dcc30882a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:13.771530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728debc-4ea2-4fb5-a9fa-5e0dcc30882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:13.913026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eaacc-30c7-4821-ba10-2bea223e28eb None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:13.968193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3728debc-4ea2-4fb5-a9fa-5e0dcc30882a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:13.971196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3728debc-4ea2-4fb5-a9fa-5e0dcc30882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:13.973915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:13.974609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:13.974886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:13.975153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:13.975918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:13.976227 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:13.977753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:13.998640 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 13/26] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:14.004261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.015671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:14.112378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 14/27] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:14.113929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50d8b4-582b-49f5-b048-e4fac66ceabb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.114219 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50d8b4-582b-49f5-b048-e4fac66ceabb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.114491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50d8b4-582b-49f5-b048-e4fac66ceabb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.115852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 15/28] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:14.144373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.144761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.145044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.169740 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:14.178264 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:14.179458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:14.180684 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b736f31-2ed5-4e4d-af15-e1f24b91ba3d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:14.232521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:14.235899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.236269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.236557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.239468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:14.241194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:14.247001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 14/29] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:14 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:14.248064 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.308510 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 16/30] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 14:20:14.316497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.398157 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac45f7a-80f0-4f74-8745-95030427e786 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:14.406967 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:14.425079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17991d62-45dd-4a92-966a-ff157a84bf33 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:14.441338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:14.442469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.442960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.443428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.444390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:14.446354 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:14.466978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2ffd2-00c0-4b47-b69a-daed6661902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:14.468104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2ffd2-00c0-4b47-b69a-daed6661902b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.468523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2ffd2-00c0-4b47-b69a-daed6661902b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.468958 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2ffd2-00c0-4b47-b69a-daed6661902b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.470397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2ffd2-00c0-4b47-b69a-daed6661902b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:14.472035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2ffd2-00c0-4b47-b69a-daed6661902b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:14.474123 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 17/31] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:14.475493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57ecc1-1f81-4a94-a37e-186b4fc42547 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.475933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57ecc1-1f81-4a94-a37e-186b4fc42547 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.476411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57ecc1-1f81-4a94-a37e-186b4fc42547 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.477821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 18/32] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:14.482727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57ecc1-1f81-4a94-a37e-186b4fc42547 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.599698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57ecc1-1f81-4a94-a37e-186b4fc42547 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:14.617324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 15/33] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:20:14 2022] POST /identity/v3/roles => generated 219 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:14.618682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.665679 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:14.670289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:14.671530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.671984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.672438 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.697400 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:14.699141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a10eaa64-96a8-4021-aea4-faf9590f4851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:20:14.723568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:14.724640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:14.725643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:20:14.725643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:20:14.725643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:20:14.725643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:14.727639 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 19/34] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:14.729880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f108df-c65f-4e76-bd6c-b1e230e07c15 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.730596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f108df-c65f-4e76-bd6c-b1e230e07c15 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.731065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f108df-c65f-4e76-bd6c-b1e230e07c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.738218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:14.753702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-77f108df-c65f-4e76-bd6c-b1e230e07c15 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:14.755062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-77f108df-c65f-4e76-bd6c-b1e230e07c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:14.783776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010a398-aed3-46f5-8d9f-5be7c44551d8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:14.825512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb2820-b368-4bef-984b-cfbee30cd12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:14.826760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb2820-b368-4bef-984b-cfbee30cd12f None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.827221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb2820-b368-4bef-984b-cfbee30cd12f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.827677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb2820-b368-4bef-984b-cfbee30cd12f None admin] PATH_INFO: `/v3/domains/default/users/575363a61b9b436f9c41ee0b0af8ab40/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.830415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb2820-b368-4bef-984b-cfbee30cd12f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=575363a61b9b436f9c41ee0b0af8ab40, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:14.832214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb2820-b368-4bef-984b-cfbee30cd12f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:14.838582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 20/35] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:14.840283 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a587b81-8f16-43c9-b080-7f73da5c2bd5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.840593 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a587b81-8f16-43c9-b080-7f73da5c2bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.841026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a587b81-8f16-43c9-b080-7f73da5c2bd5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.849289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 21/36] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:14.849289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a587b81-8f16-43c9-b080-7f73da5c2bd5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.864638 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 16/37] 10.222.0.73 () {58 vars in 1361 bytes} [Thu May 19 14:20:14 2022] PUT /identity/v3/domains/default/users/575363a61b9b436f9c41ee0b0af8ab40/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:14.866091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:14.866444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:14.866730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:14.888338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:14.890078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:14.973275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 17/38] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:14.974130 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:14.991432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:14.999480 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a587b81-8f16-43c9-b080-7f73da5c2bd5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:15.063060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:15.071541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:15.072272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.072556 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.073180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.074029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:15.074344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:15.076239 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:15.098762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bc701-b1ad-41f6-9c8a-7f47dfff814f None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:15.104656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 22/39] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:15.105683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:15.140175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:15.141113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.141421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.141697 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.142498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:15.143910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:15.160937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 18/40] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:20:14 2022] POST /identity/v3/roles => generated 225 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:15.161847 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:15.248877 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:15.250462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50bc1f-37e3-4ebe-86f3-535ed2a477c1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:15.292757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:15.293894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.294380 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.294813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db9454-7b35-4418-9a28-257648eab212 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:15.295081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.296057 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:15.297999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:15.334490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3253437-b764-4ed6-bae8-2c1addfe8102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:15.335223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253437-b764-4ed6-bae8-2c1addfe8102 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.335510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253437-b764-4ed6-bae8-2c1addfe8102 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.335789 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253437-b764-4ed6-bae8-2c1addfe8102 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.336538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3253437-b764-4ed6-bae8-2c1addfe8102 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:15.338163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3253437-b764-4ed6-bae8-2c1addfe8102 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:15.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 19/41] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:20:15 2022] POST /identity/v3/projects => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:15.381298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a2992-1025-4c8c-b669-a02130714d03 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.381298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a2992-1025-4c8c-b669-a02130714d03 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.381298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a2992-1025-4c8c-b669-a02130714d03 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.381298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 20/42] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:15.383715 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.384011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.384295 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.403521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:15.405112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:15.447514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 23/43] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:20:15 2022] POST /identity/v3/roles => generated 223 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:15.501932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 21/44] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:15.534725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:15.600299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:15.660122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aebb3-278f-46d6-b608-509d13d041c2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:15.698247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd7f-68cf-4bfa-b8c4-24f97526976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:15.699071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd7f-68cf-4bfa-b8c4-24f97526976c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.699355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd7f-68cf-4bfa-b8c4-24f97526976c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.699631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd7f-68cf-4bfa-b8c4-24f97526976c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.700919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cd7f-68cf-4bfa-b8c4-24f97526976c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:15.702683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cd7f-68cf-4bfa-b8c4-24f97526976c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:15.704293 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 24/45] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:15.709981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:15.764703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:15.814637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cf3ff-e448-4ee7-91e5-0a976aca2e4a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:15.848789 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f13abb-a8b4-4422-bb02-6d57d33fd63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:15.849486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13abb-a8b4-4422-bb02-6d57d33fd63e None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:15.849768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13abb-a8b4-4422-bb02-6d57d33fd63e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:15.850045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13abb-a8b4-4422-bb02-6d57d33fd63e None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:15.850762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f13abb-a8b4-4422-bb02-6d57d33fd63e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:15.852230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f13abb-a8b4-4422-bb02-6d57d33fd63e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:15.877759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 22/46] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:20:15 2022] POST /identity/v3/projects => generated 312 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:17.297148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26347cde-6857-467b-98ef-edecf2e5fac3 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:17.297709 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26347cde-6857-467b-98ef-edecf2e5fac3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:17.297709 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26347cde-6857-467b-98ef-edecf2e5fac3 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:17.299075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 25/47] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:17.304450 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:17.304845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:17.305004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:17.322672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:17.323859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:17.392791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 23/48] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:17.415714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26347cde-6857-467b-98ef-edecf2e5fac3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:17.523841 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26347cde-6857-467b-98ef-edecf2e5fac3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:17.544757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:17.560995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:17.561863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:17.562148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:17.562421 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:17.573537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:17.581140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:17.582208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:17.583185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 26/49] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:17.590093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:17.626554 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:17.706528 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feed00-7319-40b3-8d10-76b0bfb7ccc0 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:17.739020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:17.740136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:17.740420 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:17.740691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:17.741478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:17.741788 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:17.743201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:17.751178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 24/50] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:17.756807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:17.862758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81d3ca-d246-49e8-a3d1-c1b9cbd539b1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:17.898631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:17.899563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:17.899842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:17.900113 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:17.901600 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:17.903108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:17.905223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 27/51] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:17 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:17.911380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:18.006711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:18.014371 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6efbcf-0302-4111-9bec-fd1167f3e6d4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:18.048971 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ef9d2-b54a-4521-a2bb-5aca24f49cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:18.049811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ef9d2-b54a-4521-a2bb-5aca24f49cae None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:18.050093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ef9d2-b54a-4521-a2bb-5aca24f49cae None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:18.050369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ef9d2-b54a-4521-a2bb-5aca24f49cae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:18.051549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ef9d2-b54a-4521-a2bb-5aca24f49cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:18.053045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ef9d2-b54a-4521-a2bb-5aca24f49cae None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:18.054720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 25/52] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:18.060463 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:18.079227 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2903773-497f-48c4-9dea-bc2826dd3b92 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:18.079484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2903773-497f-48c4-9dea-bc2826dd3b92 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:18.079769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2903773-497f-48c4-9dea-bc2826dd3b92 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:18.081226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 26/53] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:18.085992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:18.086286 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:18.086557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:18.102837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:18.104019 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:18.110403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:18.164129 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8c704-d089-45d0-8246-f40bfe21e260 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:18.172938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 27/54] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:18.195381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:18.198363 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:18.199041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:18.199326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:18.199602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:18.200440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.domain.id=default, user.id=575363a61b9b436f9c41ee0b0af8ab40) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:18.200753 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:18.202247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:18.211119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 28/55] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 14:20:18 2022] GET /identity/v3/role_assignments?user.id=575363a61b9b436f9c41ee0b0af8ab40&scope.domain.id=default&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:18.293743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:18.304251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b45547-21b5-4d12-8790-76d012d62902 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:18.341614 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398579-5f2e-425c-aa3a-dfcbbe26507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:18.342434 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398579-5f2e-425c-aa3a-dfcbbe26507c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:18.342716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398579-5f2e-425c-aa3a-dfcbbe26507c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:18.342991 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398579-5f2e-425c-aa3a-dfcbbe26507c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:18.344280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398579-5f2e-425c-aa3a-dfcbbe26507c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:18.345891 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398579-5f2e-425c-aa3a-dfcbbe26507c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:18.347574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 28/56] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:18.353704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:18.463232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3b737-14ae-409f-85ca-71c71d2fdb75 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:18.475703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:18.499198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2daf9e-5747-40ef-a8ea-3425b9b114e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:18.499876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2daf9e-5747-40ef-a8ea-3425b9b114e8 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:18.500162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2daf9e-5747-40ef-a8ea-3425b9b114e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:18.500438 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2daf9e-5747-40ef-a8ea-3425b9b114e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:18.501172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2daf9e-5747-40ef-a8ea-3425b9b114e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:18.502751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2daf9e-5747-40ef-a8ea-3425b9b114e8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:18.533721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 29/57] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 14:20:18 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:20.729600 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a512b5-bb15-4dbb-9384-b0cc177c48a6 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:20.730189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a512b5-bb15-4dbb-9384-b0cc177c48a6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:20.730189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a512b5-bb15-4dbb-9384-b0cc177c48a6 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:20.731537 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 29/58] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:20.738288 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:20.738996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:20.739272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:20.768914 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:20.770465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:20.841701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 30/59] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:20.873959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a512b5-bb15-4dbb-9384-b0cc177c48a6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:20.980408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a512b5-bb15-4dbb-9384-b0cc177c48a6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:21.014935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84473da-0a36-4998-85ed-47a16957a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:21.015720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84473da-0a36-4998-85ed-47a16957a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:21.016006 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84473da-0a36-4998-85ed-47a16957a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:21.016271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84473da-0a36-4998-85ed-47a16957a2ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:21.017623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84473da-0a36-4998-85ed-47a16957a2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:21.019077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84473da-0a36-4998-85ed-47a16957a2ca None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:21.020677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 30/60] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:21.026449 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:21.122918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:21.130156 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4caba-4b5c-4241-bc18-4bfa4c10ac9b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:21.167109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08347f22-cc54-49db-8694-3dee40510eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:21.167765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08347f22-cc54-49db-8694-3dee40510eec None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:21.168042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08347f22-cc54-49db-8694-3dee40510eec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:21.168317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08347f22-cc54-49db-8694-3dee40510eec None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:21.169020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08347f22-cc54-49db-8694-3dee40510eec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:21.170509 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08347f22-cc54-49db-8694-3dee40510eec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:21.199615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 31/61] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:21 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:24.786794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250776e-9f81-4214-a521-f3415a29ec03 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:24.787702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250776e-9f81-4214-a521-f3415a29ec03 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:24.788113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250776e-9f81-4214-a521-f3415a29ec03 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:24.789935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 31/62] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:24.794949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4266d-ad33-462b-a942-ac312552fda3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:24.795716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4266d-ad33-462b-a942-ac312552fda3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:24.796145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4266d-ad33-462b-a942-ac312552fda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:24.799214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5862b4-e1a6-402a-99da-df1ab714657a None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:24.800105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5862b4-e1a6-402a-99da-df1ab714657a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:24.802207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5862b4-e1a6-402a-99da-df1ab714657a None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:24.803761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 32/63] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:24.810146 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:24.810699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:24.811194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:24.824047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b3a4266d-ad33-462b-a942-ac312552fda3 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:24.825262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b3a4266d-ad33-462b-a942-ac312552fda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:24.843599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:24.844924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:24.869690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:24.909532 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 32/64] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:24.911437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d28cbf-2462-4c1a-9ede-f8360f6b6b56 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:24.911941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d28cbf-2462-4c1a-9ede-f8360f6b6b56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:24.912411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d28cbf-2462-4c1a-9ede-f8360f6b6b56 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:24.913904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 33/65] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:24.926874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:24.927163 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:24.928080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:24.941128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 33/66] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:24.942398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:24.966383 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:24.967655 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:25.046063 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 34/67] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:25.046871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.051526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42787-fab8-4a7f-a1fe-cf1cde8348d4 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.061267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.107017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.107841 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.108126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.108401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.119925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:25.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:25.130381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 34/68] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:25.131387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.151197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.159294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316baf5-bdc7-4b7d-9e4c-7d36a315b1b2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.193458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef26036-0542-40f9-a0ca-18bbdd082f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.194299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef26036-0542-40f9-a0ca-18bbdd082f00 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.194582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef26036-0542-40f9-a0ca-18bbdd082f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.194862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef26036-0542-40f9-a0ca-18bbdd082f00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.205131 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef26036-0542-40f9-a0ca-18bbdd082f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cef26036-0542-40f9-a0ca-18bbdd082f00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:25.213448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:25.214500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:25.215281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 35/69] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:25.215935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f96b-0863-405f-9125-e1120f3a48a2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.216217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f96b-0863-405f-9125-e1120f3a48a2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.216525 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f96b-0863-405f-9125-e1120f3a48a2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.221819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 36/70] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:25.223347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427ef1a-79cf-4445-84f1-18e4b1cf753f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.223629 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427ef1a-79cf-4445-84f1-18e4b1cf753f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.223895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427ef1a-79cf-4445-84f1-18e4b1cf753f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.225295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 37/71] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:25.225924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427ef1a-79cf-4445-84f1-18e4b1cf753f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.240033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc519cc-268a-418d-bb15-4848ec7a25a5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.273698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.274507 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.274782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.275053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.285932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:25.293634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:25.294486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 35/72] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:25.295700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.300798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.336664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427ef1a-79cf-4445-84f1-18e4b1cf753f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.371099 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.371775 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.372057 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.372331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.373086 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:25.373444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.374832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99e841-5263-4496-8a83-3627d667cd0b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.375323 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.383411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 38/73] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.384656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.384963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.385263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.397400 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f22585-a595-4269-9716-cde3884643e3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.402064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:25.403269 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:25.429852 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.430526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.430809 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.431084 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.431843 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:25.432155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.433597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e450cdd-1fba-4e85-9056-29e9570845cc None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.441528 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 36/74] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.442714 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.443001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.443277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.459177 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:25.460382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:25.468600 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 39/75] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:25.469443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.527316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.538003 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 37/76] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:25.538936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.570925 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d704832-860a-4e98-a14a-0b1df2a7a3b4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.603828 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.604503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.604790 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.605065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.605844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:25.606156 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.607552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.608044 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.616913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 40/77] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.642797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ef01-beb7-4106-9411-eec15f972b98 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.678869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.679769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.680166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.680411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.695068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.697476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.699277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 38/78] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.700056 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.729422 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a10ff9-0bfc-4f0b-a649-bb783bc36cc1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.752159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.764198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.764473 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.764740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.765977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.767436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.769209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 41/79] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.769910 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.803690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293f109-cea2-4409-92cf-47bd4b8f8a1b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.829100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:25.837144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.837978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.838237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.838526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.839689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.841139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.842928 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 39/80] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.843595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.869556 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00efab-d12e-4147-8fc9-c7323ee798f1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.902357 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.903171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.903454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.903725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.916472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:25.924904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:25.926065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 42/81] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:25.926880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.942938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7653d20-f76d-4b0d-8155-67fb23718c19 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:25.977062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:25.977987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:25.978272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:25.978550 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:25.979778 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:25.981296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:25.982941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 40/82] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:25.983715 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:25.991550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.043162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9436e96-5ce9-488c-a145-b31eff8fe887 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.052087 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.077202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.077881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.078168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.078446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.090333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfb7b-bce4-439c-97c5-e0c43cda0bc4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.125475 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.126140 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.126419 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.126676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.145912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.147396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.195738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.197232 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.198022 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.246820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 43/83] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:26.247642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.258397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.298008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 41/84] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:26.299255 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.348460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a9ae-2648-436b-b03e-35bb55f6f2c0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.381046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.381744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.382017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.382291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.382996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.384468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.398777 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c69a5-d974-41a0-8c13-71eb7e0fe9a7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.403463 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 44/85] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:26 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:26.404642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.417681 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.446906 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.447609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.447898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.448184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.448968 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:26.449307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.450755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.460637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 42/86] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:26.461380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.491700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.524279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fb8b6-bf9a-4383-b869-d2b5413df557 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.563664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.564635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.564970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.565329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.569444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178071ab-e371-4493-94a2-f3f792ae8594 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.605153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.606926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.606926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.607115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.610815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.613750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.615076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 43/87] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:26.615604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.650580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.652060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.662732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.729985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.738615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8000b0-ec84-4bc7-ae3f-64f093ace39f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.771943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 45/88] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:26.772819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.781458 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.782842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.783131 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.783410 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.785022 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.786860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.788605 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 44/89] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:26.789477 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.873947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7a0f0-88ac-47cd-86ee-09eee67e3dab None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.883449 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:26.901063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78a6ab-7f47-4834-b496-46db57902241 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:26.909453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.910353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.910639 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.910916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.912142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.913646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.915309 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 46/90] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:26.916191 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.948374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:26.949055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:26.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:26.949643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:26.950592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] RBAC: Adding query filter params (role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=dbc93960a58a448881c8655e47aa3062) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:26.951005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:26.952502 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:26.958907 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 45/91] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/role_assignments?user.id=dbc93960a58a448881c8655e47aa3062&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=afd7d26260694a27a2da7aca75966a85 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:26.959746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:26.991298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:27.069426 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637a1aa-ce1c-44e1-a93c-44f593365ff0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:27.120969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:27.122304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:27.122778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:27.125042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:27.127947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:27.132482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:27.135355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 47/92] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:27.136691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:27.139981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaeed5-21af-4bed-a485-4e1283c60595 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:27.197405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:27.198896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:27.199663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:27.200173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:27.201847 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=575363a61b9b436f9c41ee0b0af8ab40) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:27.202495 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:27.204376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:27.212387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 46/93] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:26 2022] GET /identity/v3/role_assignments?user.id=575363a61b9b436f9c41ee0b0af8ab40&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:27.214324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:27.233645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:27.289346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd4e07-22e3-4234-920f-8b89706b4af1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:27.336163 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:27.337265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:27.339750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:27.340256 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:27.389182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:27.402496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd1c5-c4ae-4762-96a9-d8386f0df0b3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:27.448716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:27.449817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:27.451767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:27.452272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:27.453366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=dbc93960a58a448881c8655e47aa3062) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:27.455116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:27.456834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:27.470821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 47/94] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:27 2022] GET /identity/v3/role_assignments?user.id=dbc93960a58a448881c8655e47aa3062&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:27.485006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:27.487018 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:27.539662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:27.614445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 48/95] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:27.620346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:27.751262 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f70b5-1628-4b18-a704-1f702174dede None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:27.792422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e646e-15a3-4ea7-bcdc-21291e90c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:27.793442 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e646e-15a3-4ea7-bcdc-21291e90c723 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:27.793796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e646e-15a3-4ea7-bcdc-21291e90c723 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:27.794136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e646e-15a3-4ea7-bcdc-21291e90c723 None admin] PATH_INFO: `/v3/projects/c330cff3a2974ea4a01f3872c323f5cb` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:27.798155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e646e-15a3-4ea7-bcdc-21291e90c723 None admin] RBAC: Authorizing `identity:get_project(project_id=c330cff3a2974ea4a01f3872c323f5cb)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:27.799808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e646e-15a3-4ea7-bcdc-21291e90c723 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:27.801753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 48/96] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:27 2022] GET /identity/v3/projects/c330cff3a2974ea4a01f3872c323f5cb => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:27.807838 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:27.921740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5b4cb-e653-48e7-878c-1c56bb79b180 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:27.968825 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:27.969529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:27.969825 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:27.970104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:27.971024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=c330cff3a2974ea4a01f3872c323f5cb, user.id=dbc93960a58a448881c8655e47aa3062) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:27.971341 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:27.972819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05525f-aa7e-45e7-9c5c-a55079839ab0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:27.979149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 49/97] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:27 2022] GET /identity/v3/role_assignments?user.id=dbc93960a58a448881c8655e47aa3062&scope.project.id=c330cff3a2974ea4a01f3872c323f5cb&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:29.252494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638b423-9356-40a2-8243-0593a9cd16e9 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:29.253009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638b423-9356-40a2-8243-0593a9cd16e9 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:29.253009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638b423-9356-40a2-8243-0593a9cd16e9 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:29.254524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 49/98] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:29.259992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:29.260797 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:29.261319 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:29.278890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:29.280871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:29.283476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:29.365965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 50/99] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:29.389133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638b423-9356-40a2-8243-0593a9cd16e9 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:29.423920 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:29.517838 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638b423-9356-40a2-8243-0593a9cd16e9 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:29.570033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95634135-b711-47fe-84b5-2a4e85b42093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:29.570872 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95634135-b711-47fe-84b5-2a4e85b42093 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:29.571160 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95634135-b711-47fe-84b5-2a4e85b42093 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:29.571443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95634135-b711-47fe-84b5-2a4e85b42093 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:29.572712 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95634135-b711-47fe-84b5-2a4e85b42093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:29.574221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95634135-b711-47fe-84b5-2a4e85b42093 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:29.575878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 50/100] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:29.581625 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:29.697535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcda96-e56e-4939-8adb-a990196bd404 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:29.745753 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b1be0-96dc-4539-b9ef-ae4173ce3701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:29.746444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1be0-96dc-4539-b9ef-ae4173ce3701 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:29.746781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1be0-96dc-4539-b9ef-ae4173ce3701 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:29.747067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1be0-96dc-4539-b9ef-ae4173ce3701 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:29.747809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1be0-96dc-4539-b9ef-ae4173ce3701 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:29.749348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1be0-96dc-4539-b9ef-ae4173ce3701 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:29.763455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:29.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 51/101] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:20:29 2022] POST /identity/v3/projects => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:29.901166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe48c4f-9fd0-45f3-97f8-9774305df81a None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:29.901548 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe48c4f-9fd0-45f3-97f8-9774305df81a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:29.901811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe48c4f-9fd0-45f3-97f8-9774305df81a None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:29.903349 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 51/102] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:29.909299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:29.909600 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:29.909876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:29.931939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:29.933182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:30.020686 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 52/103] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:30.056425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe48c4f-9fd0-45f3-97f8-9774305df81a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:30.159661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:30.174156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe48c4f-9fd0-45f3-97f8-9774305df81a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:30.219870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:30.220865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.221153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.221457 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.240118 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:30.251040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:30.252113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:30.253395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 52/104] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:30.259913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:30.386969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87244759-037e-4611-9f09-6fe2943b7637 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:30.409703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:30.428837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:30.429731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.430021 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.430322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.431108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:30.431418 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:30.432944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246eb9-1c4f-4225-9984-f484c351c67d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:30.448012 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 53/105] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:30.453753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:30.579246 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584a27-ecb5-4906-9572-426b8d16ee82 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:30.629969 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:30.632680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fa6a9-e626-421a-9f49-920d15f310db None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.632971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fa6a9-e626-421a-9f49-920d15f310db None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.633262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fa6a9-e626-421a-9f49-920d15f310db None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.635109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 54/106] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:30.635328 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.635629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.635910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] PATH_INFO: `/v3/users/610d6644696c42e89d4b93f9418a107d` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.641696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e2712-ac88-4b72-8afc-0b62e6e6fb0b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.641929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e2712-ac88-4b72-8afc-0b62e6e6fb0b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.642351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e2712-ac88-4b72-8afc-0b62e6e6fb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.676108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-005e2712-ac88-4b72-8afc-0b62e6e6fb0b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:30.677353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-005e2712-ac88-4b72-8afc-0b62e6e6fb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:30.682046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] RBAC: Authorizing `identity:get_user(user_id=610d6644696c42e89d4b93f9418a107d)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:30.683598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:30.689105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 53/107] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:30 2022] GET /identity/v3/users/610d6644696c42e89d4b93f9418a107d => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:30.693610 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:30.712244 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:30.796705 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 55/108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:30.805259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f914b-5075-4c08-8dae-5a1f854dcc9f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.805708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f914b-5075-4c08-8dae-5a1f854dcc9f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.805979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f914b-5075-4c08-8dae-5a1f854dcc9f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.810118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 56/109] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:30.811171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce599fc-6309-457f-a794-9592588634d8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.811462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce599fc-6309-457f-a794-9592588634d8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.811817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce599fc-6309-457f-a794-9592588634d8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.820602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 57/110] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:30.821957 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c1f5-25db-468b-9ef8-d14a99c7e68b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.822240 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c1f5-25db-468b-9ef8-d14a99c7e68b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.822516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c1f5-25db-468b-9ef8-d14a99c7e68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.839766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c3c5c1f5-25db-468b-9ef8-d14a99c7e68b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:30.840996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c3c5c1f5-25db-468b-9ef8-d14a99c7e68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:30.849356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca88c-74c3-434b-a64d-a061403586e5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:30.888306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:30.889930 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.890456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.890892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.917271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 58/111] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:30.917271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:30.917271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:30.917271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:30.930180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:30.931549 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:30.976140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:30.992504 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:30.994387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.005168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.017659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 59/112] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:31.018714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.121987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 54/113] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:31.122989 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.123156 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff463b4-ddb1-498b-9b5c-01d5bea7615a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.160248 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.161361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.161903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.162325 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.175450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:31.183338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:31.184206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:31.184982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:31.185610 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 60/114] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:31.185863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.222391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.243660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850a9e-8349-4a51-a6a3-05086beed357 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.254606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.293782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.294672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.295276 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.295568 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.311168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:31.312809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c8483-9051-4b55-baa0-9868153aba36 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:31.321164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:31.322045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:31.322926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 55/115] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:31.323566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.365822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.366659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.366935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.367202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.383000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:31.394866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:31.396361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:31.397332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:31.399786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 61/116] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:31.400712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.447024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b9d57-4b37-473b-a420-9ab0ff51bfa4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.489629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.490774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.491054 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.491354 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.493295 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:31.494773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.496490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 56/117] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:31.497688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.519102 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.530016 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf2be8-20fc-46dd-af9a-32443077a242 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.535917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.565321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.566266 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.566546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.566873 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.567637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:31.567997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:31.569437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.578337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 62/118] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:31.578974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.609808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a20cbd-ac3c-44d3-af97-d260cea51921 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.646076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.646823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.647044 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.647319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.648108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:31.648393 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:31.649936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd11c57-a840-4f31-b118-b94f7ebe413c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.663536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 57/119] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:31.663536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412111b-5923-49ba-8616-5e798141ce9a None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.663536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412111b-5923-49ba-8616-5e798141ce9a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.663536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412111b-5923-49ba-8616-5e798141ce9a None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.663536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 58/120] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:31.663536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412111b-5923-49ba-8616-5e798141ce9a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.690549 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc5117-641a-4b15-a213-c3e4c5f0ddeb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.725120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.725749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.726035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.726307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.727061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:31.727374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:31.728771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.737281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 63/121] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:31.738072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.760081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.776993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:31.781903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412111b-5923-49ba-8616-5e798141ce9a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.816651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.817431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.817776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.818115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.819011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] RBAC: Adding query filter params (role.id=7366d82e48f645008e175b0e2fe79ce1, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=610d6644696c42e89d4b93f9418a107d) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:31.819392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:31.821007 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.827334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 59/122] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/role_assignments?user.id=610d6644696c42e89d4b93f9418a107d&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=7366d82e48f645008e175b0e2fe79ce1 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:31.833993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:31.857707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245ea44-24e5-4319-a12d-7ff54c82fc1e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:31.906259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84c739-379b-4855-b4fb-4fa8bf0d2585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:31.907919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84c739-379b-4855-b4fb-4fa8bf0d2585 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.908676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84c739-379b-4855-b4fb-4fa8bf0d2585 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.909675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84c739-379b-4855-b4fb-4fa8bf0d2585 None admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.911599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84c739-379b-4855-b4fb-4fa8bf0d2585 None admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:31.913591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84c739-379b-4855-b4fb-4fa8bf0d2585 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:31.921161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 64/123] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:31.921161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:31.921161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:31.921161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:31.946663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:31.948301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:31.964127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77277-f4cd-4816-a99c-a5aa06041155 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.008379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.009207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.009544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.009817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.011560 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.013436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.015455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 60/124] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:31 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:32.021095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.053790 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 65/125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:32.053790 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.072345 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.077343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.153226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa15e2a-5e49-49e8-a15e-3d795a4709da None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.164297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a3fe1-7c0c-4706-8a44-403d8fbfd880 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.193222 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.193747 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.194252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.195734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.197661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.199692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 61/126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:32.200874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.204267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.206060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.206060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.206060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.207195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.208657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.214918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 66/127] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:32.214918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.293397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.311715 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c9f8-da12-48e4-a215-aa938cf45aed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.315681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.320546 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b805916-93b4-4ffe-89ea-3ac51bb015db None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.348455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.349302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.349592 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.349849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.360597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.360943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:32.362078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.362564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.363050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.364547 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.366502 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.368644 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 62/128] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:32.369827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:32.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:32.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:32.375793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 67/129] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:32.376570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.487835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7429-7606-4d56-bf2d-89f4aa0b4e0c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.497227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b281a1-d7e0-4a49-8346-079de4e44be0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.524993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.526443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.526824 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.527198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.528505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.530725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.532746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 63/130] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:32.540088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.542723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.545428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.545939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.546415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5/users/575363a61b9b436f9c41ee0b0af8ab40/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.548799 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] RBAC: Authorizing `identity:create_grant(project_id=730c37dd34844384872ab44ea74c7db5, user_id=575363a61b9b436f9c41ee0b0af8ab40, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.550654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.573395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 68/131] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:32 2022] PUT /identity/v3/projects/730c37dd34844384872ab44ea74c7db5/users/575363a61b9b436f9c41ee0b0af8ab40/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:32.574617 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.575342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.580204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.659141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9fc9-92f7-42c3-b427-4114faed8c57 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.698388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.699343 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.699699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.699976 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5/users/dbc93960a58a448881c8655e47aa3062/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.702628 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=730c37dd34844384872ab44ea74c7db5, user_id=dbc93960a58a448881c8655e47aa3062, role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.704309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.721300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac00cab-5426-4125-968c-f206b0d8d929 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.722752 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 64/132] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:32 2022] PUT /identity/v3/projects/730c37dd34844384872ab44ea74c7db5/users/dbc93960a58a448881c8655e47aa3062/roles/afd7d26260694a27a2da7aca75966a85 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:32.723823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.803486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.803704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.803934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.804168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.804976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:32.806974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.808004 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14715a-3466-40ba-bd5f-d1bb4afa560f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.819802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 69/133] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:32.821580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216dbc-6354-4b85-ab88-4892df10897e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.822072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216dbc-6354-4b85-ab88-4892df10897e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.822544 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216dbc-6354-4b85-ab88-4892df10897e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.824171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 70/134] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:32.826913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216dbc-6354-4b85-ab88-4892df10897e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:32.900028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.918361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537142a9-d6f3-46d4-8c1c-5e54e342e7a5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:32.952374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:32.964870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f8881-c51b-4290-88dc-35d10af7aa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:32.966139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8881-c51b-4290-88dc-35d10af7aa6a None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.966431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8881-c51b-4290-88dc-35d10af7aa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.966705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8881-c51b-4290-88dc-35d10af7aa6a None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5/users/dbc93960a58a448881c8655e47aa3062/roles/9f78fe1c53c24d8ca59e298f2c585211` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:32.969927 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f8881-c51b-4290-88dc-35d10af7aa6a None admin] RBAC: Authorizing `identity:create_grant(project_id=730c37dd34844384872ab44ea74c7db5, user_id=dbc93960a58a448881c8655e47aa3062, role_id=9f78fe1c53c24d8ca59e298f2c585211)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:32.971463 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f8881-c51b-4290-88dc-35d10af7aa6a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:32.989019 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 65/135] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:32 2022] PUT /identity/v3/projects/730c37dd34844384872ab44ea74c7db5/users/dbc93960a58a448881c8655e47aa3062/roles/9f78fe1c53c24d8ca59e298f2c585211 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:32.990896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:32.991238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:32.991522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:33.026970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216dbc-6354-4b85-ab88-4892df10897e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:33.032089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:33.034908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:33.099303 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:33.101526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:33.104282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:33.106966 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:33.114212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:33.115729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:33.121288 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 71/136] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:32 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:33.128650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:33.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 66/137] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:33.200822 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:33.327251 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3bdd1-2bd2-43de-a94b-76194f9dcd28 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:33.346829 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:33.367075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:33.375370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3eb5-36a4-4107-90cb-84485c24f527 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:33.376936 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:33.377920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:33.378335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:33.378619 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] PATH_INFO: `/v3/projects/c330cff3a2974ea4a01f3872c323f5cb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:33.386029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] RBAC: Authorizing `identity:get_project(project_id=c330cff3a2974ea4a01f3872c323f5cb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:33.387517 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:33.389547 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 72/138] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:33 2022] GET /identity/v3/projects/c330cff3a2974ea4a01f3872c323f5cb => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:33.395431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:33.431595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:33.432419 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:33.432701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:33.432977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:33.437984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:33.439507 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:33.441446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 67/139] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:33.446462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:33.587954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df667-a159-4366-82de-7fcd400a0129 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:33.592629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb82a3a-3855-4388-a409-46c0a3d37f3e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:33.654995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1a04-caf2-4931-ae2a-e3dcd207aa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:33.656101 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1a04-caf2-4931-ae2a-e3dcd207aa0f None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:33.656581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1a04-caf2-4931-ae2a-e3dcd207aa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:33.657053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1a04-caf2-4931-ae2a-e3dcd207aa0f None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:33.658354 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c1a04-caf2-4931-ae2a-e3dcd207aa0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:33.660259 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c1a04-caf2-4931-ae2a-e3dcd207aa0f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:33.662477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec37e1c-69c8-449e-85f3-c248c3d9a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:33.669457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37e1c-69c8-449e-85f3-c248c3d9a1ef None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:33.669457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37e1c-69c8-449e-85f3-c248c3d9a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:33.669457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37e1c-69c8-449e-85f3-c248c3d9a1ef None admin] PATH_INFO: `/v3/projects/c330cff3a2974ea4a01f3872c323f5cb/users/dbc93960a58a448881c8655e47aa3062/roles/9f78fe1c53c24d8ca59e298f2c585211` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:33.682200 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec37e1c-69c8-449e-85f3-c248c3d9a1ef None admin] RBAC: Authorizing `identity:create_grant(project_id=c330cff3a2974ea4a01f3872c323f5cb, user_id=dbc93960a58a448881c8655e47aa3062, role_id=9f78fe1c53c24d8ca59e298f2c585211)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:33.683896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec37e1c-69c8-449e-85f3-c248c3d9a1ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:33.724000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 73/140] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:33 2022] PUT /identity/v3/projects/c330cff3a2974ea4a01f3872c323f5cb/users/dbc93960a58a448881c8655e47aa3062/roles/9f78fe1c53c24d8ca59e298f2c585211 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:20:33.730636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 68/141] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:33 2022] POST /identity/v3/users => generated 283 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:20:34.468597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6a02-eb78-43bb-ac3f-a51138f683cf None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:34.468980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6a02-eb78-43bb-ac3f-a51138f683cf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:34.469151 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6a02-eb78-43bb-ac3f-a51138f683cf None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:34.470826 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 74/142] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:34.475811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:34.476245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:34.476428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:34.502287 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:34.503507 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:34.505806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:34.588857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 69/143] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:34.611786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6a02-eb78-43bb-ac3f-a51138f683cf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:34.674944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:34.717828 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6a02-eb78-43bb-ac3f-a51138f683cf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:34.754342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:34.755085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:34.755367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:34.755643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:34.766767 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:34.774543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:34.775484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:34.776261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 75/144] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:34.782004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:34.899028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b1d16-9eee-4f6b-b4cb-d5b47b7396dd None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:34.940271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:34.940271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:34.940271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:34.940271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:34.940602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:34.940602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:34.942032 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:34.951128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 70/145] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:34.953623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:34.962375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:35.090973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dce865-4619-404c-bce0-5716fd7c0a45 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:35.117726 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:35.129680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:35.130623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:35.131130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:35.131405 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] PATH_INFO: `/v3/users/610d6644696c42e89d4b93f9418a107d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:35.132613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] RBAC: Authorizing `identity:get_user(user_id=610d6644696c42e89d4b93f9418a107d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:35.134113 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:35.135759 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 76/146] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:34 2022] GET /identity/v3/users/610d6644696c42e89d4b93f9418a107d => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:35.141272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:35.285465 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f67864-150c-455b-9695-2d917406a6c1 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:35.325344 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c8448-573a-4cbd-a3a8-8a63b3d6728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:35.326276 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c8448-573a-4cbd-a3a8-8a63b3d6728e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:35.326553 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c8448-573a-4cbd-a3a8-8a63b3d6728e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:35.326825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c8448-573a-4cbd-a3a8-8a63b3d6728e None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:35.328097 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c8448-573a-4cbd-a3a8-8a63b3d6728e None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:35.329681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c8448-573a-4cbd-a3a8-8a63b3d6728e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:35.331627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 71/147] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:35 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:35.337137 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:35.464812 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2e14f-5bc4-4d0f-8b16-6f67e238dc59 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:35.500978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3064a26-026d-4363-a21c-79fd05a03528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:35.502008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3064a26-026d-4363-a21c-79fd05a03528 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:35.502293 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3064a26-026d-4363-a21c-79fd05a03528 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:35.502567 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3064a26-026d-4363-a21c-79fd05a03528 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5/users/610d6644696c42e89d4b93f9418a107d/roles/7366d82e48f645008e175b0e2fe79ce1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:35.504545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3064a26-026d-4363-a21c-79fd05a03528 None admin] RBAC: Authorizing `identity:create_grant(project_id=730c37dd34844384872ab44ea74c7db5, user_id=610d6644696c42e89d4b93f9418a107d, role_id=7366d82e48f645008e175b0e2fe79ce1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:35.506026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3064a26-026d-4363-a21c-79fd05a03528 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:35.526287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 77/148] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:35 2022] PUT /identity/v3/projects/730c37dd34844384872ab44ea74c7db5/users/610d6644696c42e89d4b93f9418a107d/roles/7366d82e48f645008e175b0e2fe79ce1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:36.260420 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f759fb-9c1e-449d-b0be-7b88cd715108 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.261284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f759fb-9c1e-449d-b0be-7b88cd715108 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.261699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f759fb-9c1e-449d-b0be-7b88cd715108 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.263221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 72/149] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:36.268640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.269424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.269842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.294456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:36.295943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:36.331427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0a587-8ff4-4b33-9d0f-f05009253242 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.331949 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0a587-8ff4-4b33-9d0f-f05009253242 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.332365 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0a587-8ff4-4b33-9d0f-f05009253242 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.333915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 73/150] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:36.341009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.341595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.342024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.369917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:36.373695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:36.394086 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:36.446688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 78/151] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:36.493960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:36.499561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:36.503334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 74/152] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:36.555814 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:36.673679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01d02a-7960-4367-95ee-769beca5c8bc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:36.718003 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc62109-dde4-44e6-9886-ada1968b7dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:36.718847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc62109-dde4-44e6-9886-ada1968b7dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.719120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc62109-dde4-44e6-9886-ada1968b7dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.719393 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc62109-dde4-44e6-9886-ada1968b7dd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.750193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc62109-dde4-44e6-9886-ada1968b7dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8cc62109-dde4-44e6-9886-ada1968b7dd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:36.758392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:36.759274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:36.760047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:36.762462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 79/153] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:20:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:36.769453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203290f1-d00f-4f68-ad45-9e8eac7dd6fd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.770027 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203290f1-d00f-4f68-ad45-9e8eac7dd6fd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.770486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203290f1-d00f-4f68-ad45-9e8eac7dd6fd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.772121 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 80/154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:36.774023 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203290f1-d00f-4f68-ad45-9e8eac7dd6fd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:36.778229 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c303e8d-9024-4772-8c13-3d6264df65ed None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:36.867476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:36.886741 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c5c16-30d4-4b92-b62a-7a1e4641e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:36.888171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5c16-30d4-4b92-b62a-7a1e4641e16c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.888625 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5c16-30d4-4b92-b62a-7a1e4641e16c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.889094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5c16-30d4-4b92-b62a-7a1e4641e16c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.925867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c5c16-30d4-4b92-b62a-7a1e4641e16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cd4c5c16-30d4-4b92-b62a-7a1e4641e16c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:36.933692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:36.934589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:36.935562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 75/155] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:36.936504 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7742a91-6807-4080-a94f-739984185f23 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:36.936942 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7742a91-6807-4080-a94f-739984185f23 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:36.942013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7742a91-6807-4080-a94f-739984185f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:36.975771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-d7742a91-6807-4080-a94f-739984185f23 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:36.977199 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-d7742a91-6807-4080-a94f-739984185f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:37.017932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203290f1-d00f-4f68-ad45-9e8eac7dd6fd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:37.085938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:37.087100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:37.087578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:37.088143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:37.089128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:37.089676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:37.091406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:37.108528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 81/156] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:20:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:37.109571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:37.111194 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:37.144211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 76/157] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:37.145461 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7742a91-6807-4080-a94f-739984185f23 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:37.332489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:37.339725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd999-aa4d-4dbf-a929-b35981a61e99 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:37.394672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7742a91-6807-4080-a94f-739984185f23 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:37.433303 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:37.434253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:37.434664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:37.435076 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:37.435962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:37.436399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:37.442017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:37.456449 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 82/158] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:37.457612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:37.481954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:37.483218 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:37.483713 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:37.484175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:37.485719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:37.487399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:37.489283 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 77/159] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:37 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:37.490258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:37.630268 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:37.651974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4b474-882f-4079-97ee-e901bbb353d9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:37.716816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:37.717663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:37.717855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23a81a-685f-4f56-8857-996f51f8428d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:37.717949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:37.718204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:37.743337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:37.751752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:37.752805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:37.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:37.758679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 83/160] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:37.759478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:37.784075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb9dd0-527a-4211-a211-ba8ed7905446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:37.785015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9dd0-527a-4211-a211-ba8ed7905446 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:37.785344 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9dd0-527a-4211-a211-ba8ed7905446 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:37.785606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9dd0-527a-4211-a211-ba8ed7905446 None admin] PATH_INFO: `/v3/users/575363a61b9b436f9c41ee0b0af8ab40` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:37.787123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb9dd0-527a-4211-a211-ba8ed7905446 None admin] RBAC: Authorizing `identity:get_user(user_id=575363a61b9b436f9c41ee0b0af8ab40)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:37.788604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb9dd0-527a-4211-a211-ba8ed7905446 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:37.794546 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 78/161] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:37 2022] GET /identity/v3/users/575363a61b9b436f9c41ee0b0af8ab40 => generated 247 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:37.795926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766515a-4b4a-43e9-8291-53d227406e34 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:37.796252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766515a-4b4a-43e9-8291-53d227406e34 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:37.796531 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766515a-4b4a-43e9-8291-53d227406e34 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:37.798151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 79/162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:37.798813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766515a-4b4a-43e9-8291-53d227406e34 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:37.829928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:38.024284 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414c45-c8a4-444e-bb68-e3b6ed5c093b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:38.057785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766515a-4b4a-43e9-8291-53d227406e34 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:38.086782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef47e-6223-43d1-bdd7-ad595c977ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:38.087455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef47e-6223-43d1-bdd7-ad595c977ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.087737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef47e-6223-43d1-bdd7-ad595c977ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.088011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef47e-6223-43d1-bdd7-ad595c977ea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.114349 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:38.142888 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:38.143570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.143841 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.144106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.144856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:38.145156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:38.146569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:38.167780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 80/163] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:38.168732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:38.225000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef47e-6223-43d1-bdd7-ad595c977ea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:38.226573 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef47e-6223-43d1-bdd7-ad595c977ea3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:38.272874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:38.343579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c63a02-6012-4dda-84e4-62a06bf268a8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:38.390047 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7cfc9-c8e0-4630-a5d5-634265ec4a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:38.390908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7cfc9-c8e0-4630-a5d5-634265ec4a48 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.391196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7cfc9-c8e0-4630-a5d5-634265ec4a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.391469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7cfc9-c8e0-4630-a5d5-634265ec4a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.461573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:38.466189 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 84/164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:38.467618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.468620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.468913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.496548 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7cfc9-c8e0-4630-a5d5-634265ec4a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:38.498001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7cfc9-c8e0-4630-a5d5-634265ec4a48 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:38.510635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:38.511958 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:38.615322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 85/165] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:38.616185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:38.699430 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 81/166] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:38.700874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536189-8870-4a17-b467-6f9c5b087782 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.701160 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536189-8870-4a17-b467-6f9c5b087782 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.701459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536189-8870-4a17-b467-6f9c5b087782 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.702856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 82/167] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:38.703622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1536189-8870-4a17-b467-6f9c5b087782 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:38.720363 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:38.808286 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8e0ad-cb29-41a7-ab8e-5013c836a214 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:38.828958 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:38.863830 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1536189-8870-4a17-b467-6f9c5b087782 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:38.877822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:38.879018 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.879439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.879844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.881190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:38.882839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:38.884714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 86/168] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:38 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:38.885650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:38.908637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b850d6-3c5b-46dd-863d-4d270ea86c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:38.909941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b850d6-3c5b-46dd-863d-4d270ea86c32 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.910387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b850d6-3c5b-46dd-863d-4d270ea86c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.911075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b850d6-3c5b-46dd-863d-4d270ea86c32 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.912426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b850d6-3c5b-46dd-863d-4d270ea86c32 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:38.914496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b850d6-3c5b-46dd-863d-4d270ea86c32 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:38.916793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 83/169] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:38 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:38.918637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:38.919210 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:38.919624 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:38.940528 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:38.942456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:38.998826 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73fff3-6dcb-4447-beba-b388398e5cf5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.021047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.024531 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 84/170] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:39.025763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.039037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e8ccb-9d2e-43ec-9af1-6636ceb064a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.040410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e8ccb-9d2e-43ec-9af1-6636ceb064a2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.040907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e8ccb-9d2e-43ec-9af1-6636ceb064a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.041465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e8ccb-9d2e-43ec-9af1-6636ceb064a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.054734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e8ccb-9d2e-43ec-9af1-6636ceb064a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-883e8ccb-9d2e-43ec-9af1-6636ceb064a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:39.063203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:39.064267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 87/171] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:39.070134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dcb0-3847-468f-a03c-41e9df355f34 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.070839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dcb0-3847-468f-a03c-41e9df355f34 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.070839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dcb0-3847-468f-a03c-41e9df355f34 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.070839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 88/172] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:39.070839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dcb0-3847-468f-a03c-41e9df355f34 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.122941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.139326 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795551e-8c27-4afa-b98d-1a0667fbcd84 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.176063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.176939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.177217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.177563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.178663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.178989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dcb0-3847-468f-a03c-41e9df355f34 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.180122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.182248 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 85/173] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:39.182840 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.218597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056e65c-7fe2-42ed-912e-30fbce2c364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.219268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056e65c-7fe2-42ed-912e-30fbce2c364d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.219542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056e65c-7fe2-42ed-912e-30fbce2c364d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.219815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056e65c-7fe2-42ed-912e-30fbce2c364d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.267478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.282940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbe1f-3479-4c0f-972b-a39fa813142e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.286678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056e65c-7fe2-42ed-912e-30fbce2c364d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.288153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056e65c-7fe2-42ed-912e-30fbce2c364d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.315033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.315610 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.315887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.316153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.317003 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] RBAC: Adding query filter params (role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=dbc93960a58a448881c8655e47aa3062) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:39.317329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.318761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d772e3e-bb0f-47d1-8c5f-388e253aa6e9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.329062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 86/174] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/role_assignments?user.id=dbc93960a58a448881c8655e47aa3062&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=afd7d26260694a27a2da7aca75966a85 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:39.330634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6312c-e2e2-45d8-a4ae-1378cd8eb902 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.330917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6312c-e2e2-45d8-a4ae-1378cd8eb902 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.331186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6312c-e2e2-45d8-a4ae-1378cd8eb902 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.332568 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 87/175] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:39.333161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6312c-e2e2-45d8-a4ae-1378cd8eb902 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.358557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.411512 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 89/176] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:39.413029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.413328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.413603 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.431282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:39.432496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:39.458256 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6312c-e2e2-45d8-a4ae-1378cd8eb902 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.494896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.496495 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.496960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.497339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.504601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 90/177] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:39.506976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.510446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:39.518446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:39.519386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:39.520201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:39.524258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 88/178] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:39.524719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.525185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.613196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.624348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa32c9-21e2-4acc-b3b6-43d71d379b80 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.637284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7a8a-c948-46a6-aa20-6ebeaa6440e5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.658643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.659322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.659600 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.659925 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.660675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:39.660977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.662420 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fe980-5f15-40ea-a93c-ab08584f85bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.670135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.670302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 91/179] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:39.670795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.671073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.671345 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.671496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b286d-3298-427d-af93-d879cacc1c73 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.671778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b286d-3298-427d-af93-d879cacc1c73 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.672050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b286d-3298-427d-af93-d879cacc1c73 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.672175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=575363a61b9b436f9c41ee0b0af8ab40) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:39.672489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.673569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 92/180] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:39.674458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.674736 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.675008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.677973 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.684119 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 89/181] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/role_assignments?user.id=575363a61b9b436f9c41ee0b0af8ab40&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:39.684912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.693932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:39.695402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:39.777313 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 93/182] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:39.782836 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.786096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.800218 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d7f1b-5cf9-4127-a45e-28d3055b7f76 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.840800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.841728 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.842003 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.842259 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.843530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.845117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.846951 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 90/183] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:39.847825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.864140 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:39.908172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92266b33-f655-458d-91ac-6f27a9c46d3d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:39.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:39.952083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:39.952274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:39.952533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:39.953367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:39.953670 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:39.955094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:39.964331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 94/184] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:39.966199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:39.972540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2960dd-98fe-4731-86c8-c0868d39e2fb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.004758 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b48f-a046-43bf-abed-6f520e1120be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.005612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b48f-a046-43bf-abed-6f520e1120be None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.005889 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b48f-a046-43bf-abed-6f520e1120be None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.006159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b48f-a046-43bf-abed-6f520e1120be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.015893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118b48f-a046-43bf-abed-6f520e1120be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-4118b48f-a046-43bf-abed-6f520e1120be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:40.024024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:40.024990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:40.026175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 91/185] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:40.027024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.027024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.027024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.028791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.044133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:40.045353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:40.069141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa81a54-2dc6-4e43-8a6d-cfc8e161a8eb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.102461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.103352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.103633 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.103905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] PATH_INFO: `/v3/users/dbc93960a58a448881c8655e47aa3062` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.104458 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.105320 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] RBAC: Authorizing `identity:get_user(user_id=dbc93960a58a448881c8655e47aa3062)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.107024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.108677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 95/186] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:39 2022] GET /identity/v3/users/dbc93960a58a448881c8655e47aa3062 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:40.113489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.113573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 92/187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:40.113573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.211871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdbd4e-cee7-4653-bad6-fbe52bbd54ae None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.213336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5d4ee-1f25-415c-bccb-235acd1651d7 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.240944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.246662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.247477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.247713 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.247857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.248007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.248369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.248651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.248913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.249159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.249777 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=dbc93960a58a448881c8655e47aa3062) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:40.250082 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.250638 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.251559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.252278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 96/188] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:40.255054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.265850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 93/189] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/role_assignments?user.id=dbc93960a58a448881c8655e47aa3062&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:40.265850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.335116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.375440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dc400-5254-4209-89a9-12b2f2589300 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.378935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14a78-6964-46cb-8933-52e383ee6c45 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.414296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.415198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.415474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.415749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] PATH_INFO: `/v3/users/2ba4c2698c1b472da92acaa8a25e456a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.416970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.417691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.417977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.418243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.419009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:40.419314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.420745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.430492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 94/190] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:40.431266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.439573 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] RBAC: Authorizing `identity:get_user(user_id=2ba4c2698c1b472da92acaa8a25e456a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.441091 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.444988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 97/191] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/users/2ba4c2698c1b472da92acaa8a25e456a => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:40.444988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.511640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.552922 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325133a-568d-4b9e-b79e-c175baf8943e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.553461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d381f-0c9a-45ee-a6e0-31e792552e5c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.574883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.587751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.588291 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.588557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.588832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.588962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.589066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.589150 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.589484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.599191 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:40.606862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:40.607758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:40.608581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 98/192] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:40.609282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.663302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.664793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.713483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fb0d9-77cb-4f4d-94e3-8a616cf51c84 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.734056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.749489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.750161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.750441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.750715 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.751481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.752947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.760442 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 95/193] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:40.762271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.775276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 99/194] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:40 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:40.776937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.788785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:40.874378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fc9e0-4269-4f2a-9e1b-a5c4b4226cd0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.891159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b8ca-3c40-455f-9c9c-c284997fb920 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:40.912756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.914032 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.914314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.914590 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] PATH_INFO: `/v3/users/2ba4c2698c1b472da92acaa8a25e456a` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.919220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] RBAC: Authorizing `identity:get_user(user_id=2ba4c2698c1b472da92acaa8a25e456a)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:40.921272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:40.928293 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 96/195] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/users/2ba4c2698c1b472da92acaa8a25e456a => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:40.928293 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:40.934386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:40.935062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:40.935470 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:40.935743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:40.985919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.015530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.017003 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.053522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.065938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306fa7-136c-49a5-80c5-dfc05689c668 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.111792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.113077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.113077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.113277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.114827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:41.114827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.115705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.124734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 97/196] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:41.127136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.144551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 100/197] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:41.149748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.234131 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.248743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2dff9-6b13-4fbf-97d9-aacdf80a4bdf None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.259961 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025d406-d185-4755-9c48-0f7f81699ff3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.288085 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.289359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.289962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.290423 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] PATH_INFO: `/v3/projects/c330cff3a2974ea4a01f3872c323f5cb` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.291915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] RBAC: Authorizing `identity:get_project(project_id=c330cff3a2974ea4a01f3872c323f5cb)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.293959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.295604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.296219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 98/198] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/projects/c330cff3a2974ea4a01f3872c323f5cb => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:41.296624 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.296906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.297185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.297887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.330963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.366900 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.368390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.432415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946025e-a3fb-41fd-a1ca-d338aa2d343e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.442612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.468833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.470430 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 101/199] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:41.470708 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.471110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.471394 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.471910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] PATH_INFO: `/v3/users/610d6644696c42e89d4b93f9418a107d` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.481265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] RBAC: Authorizing `identity:get_user(user_id=610d6644696c42e89d4b93f9418a107d)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.481265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.483074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 99/200] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/users/610d6644696c42e89d4b93f9418a107d => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:41.483857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.587135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c7fcb-e35e-4205-9a2c-e8c56c2c0821 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.595297 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.610753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33ab18-7f54-4fdf-ac0a-8fe02474d302 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.622650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.623532 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.623810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.624083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.625297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.626832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.628718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 102/201] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:41.629342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.656832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.657841 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.658539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.658820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.659959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=c330cff3a2974ea4a01f3872c323f5cb, user.id=dbc93960a58a448881c8655e47aa3062) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:41.660295 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.662848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.677355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 100/202] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/role_assignments?user.id=dbc93960a58a448881c8655e47aa3062&scope.project.id=c330cff3a2974ea4a01f3872c323f5cb&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:41.678374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.685568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.795380 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79197f-78c5-4d71-a090-8c05e83b1a02 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.836452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134772c-c054-4290-b4e7-13c3c077b937 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:41.877875 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.878557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.878837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.879114 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.895120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:41.896031 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:41.896303 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:41.896566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:41.897795 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.899275 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:41.901029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 103/203] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:41.910384 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:41.912832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:41.982437 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:41.984382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:42.044647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0bf7-f755-45e7-8fc5-5db80af07e67 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:42.056566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:42.082407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:42.083417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:42.083892 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:42.084358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:42.086300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=2ba4c2698c1b472da92acaa8a25e456a) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:42.086862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:42.088570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:42.095074 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 104/204] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/role_assignments?user.id=2ba4c2698c1b472da92acaa8a25e456a&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:42.096122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:42.126751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 101/205] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:42.132761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:42.227456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:42.270450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f966b5-d065-4510-a969-eb9231d48579 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:42.292670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb254-275e-4185-968d-d5e51a6194cd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:42.312928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:42.313636 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:42.314044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:42.314313 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:42.315161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] RBAC: Adding query filter params (role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=2ba4c2698c1b472da92acaa8a25e456a) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:42.315473 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:42.316948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:42.323721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 105/206] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:42 2022] GET /identity/v3/role_assignments?user.id=2ba4c2698c1b472da92acaa8a25e456a&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=afd7d26260694a27a2da7aca75966a85 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:42.378060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8877-f89f-4633-8a58-2401f6e5000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:42.378971 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8877-f89f-4633-8a58-2401f6e5000a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:42.379248 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8877-f89f-4633-8a58-2401f6e5000a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:42.379520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8877-f89f-4633-8a58-2401f6e5000a None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:42.380783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8877-f89f-4633-8a58-2401f6e5000a None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:42.382392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8877-f89f-4633-8a58-2401f6e5000a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:42.384669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 102/207] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:42 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:42.390711 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:42.461664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:42.520649 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f70280-42bf-4be0-b6e4-6e074e25af29 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:42.570258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:42.570931 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:42.571207 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:42.571478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:42.572321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] RBAC: Adding query filter params (role.id=7366d82e48f645008e175b0e2fe79ce1, scope.project.id=730c37dd34844384872ab44ea74c7db5, user.id=610d6644696c42e89d4b93f9418a107d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:42.572684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:42.574218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1936f1-4e62-46e4-b122-ad723bcfdbe2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:42.581350 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 106/208] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:42 2022] GET /identity/v3/role_assignments?user.id=610d6644696c42e89d4b93f9418a107d&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=7366d82e48f645008e175b0e2fe79ce1 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:43.252660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fede-acd8-4080-ac94-7a6616dde719 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.253152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fede-acd8-4080-ac94-7a6616dde719 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.253152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fede-acd8-4080-ac94-7a6616dde719 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.254528 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 103/209] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:43.260196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c2a31-2a46-44ed-9ef0-b4bfca6472de None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.260635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c2a31-2a46-44ed-9ef0-b4bfca6472de None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.260810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c2a31-2a46-44ed-9ef0-b4bfca6472de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.277818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aa7c2a31-2a46-44ed-9ef0-b4bfca6472de None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:43.279354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aa7c2a31-2a46-44ed-9ef0-b4bfca6472de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:43.347634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 107/210] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:43.370327 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8fede-acd8-4080-ac94-7a6616dde719 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:43.410733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2eff-6c36-4b78-a235-7a5a97c8b700 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.411355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2eff-6c36-4b78-a235-7a5a97c8b700 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.411819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2eff-6c36-4b78-a235-7a5a97c8b700 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.413589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 108/211] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:43.418882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.419165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.419442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.436698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:43.437987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:43.468253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:43.484346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8fede-acd8-4080-ac94-7a6616dde719 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:43.507779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:43.511191 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 109/212] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:43.521411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:43.522621 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.523116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.523603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.536197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:43.539498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:43.547683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:43.548626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:43.549481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 104/213] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:43.556413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:43.643598 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4c45-18d1-4e42-8a5b-64d8b3d161d9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:43.659565 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d00d3-5e44-4ddc-b8a3-273ee8ecee7f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:43.680835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:43.682072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.682651 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.683240 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.684649 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:43.686548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:43.688898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 110/214] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:43.694122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:43.700710 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:43.701460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.701739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.702011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.702775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:43.703080 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:43.707024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:43.722701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 105/215] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:43.723676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:43.729524 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:43.787689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:43.812259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94644071-0fce-478d-8959-bd702f1457f2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:43.850068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:43.851094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.851579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.852189 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.853066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:43.854558 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:43.855190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44cc6-7809-4969-90d2-ab69d05e0613 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:43.879678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 111/216] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:43 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:20:43.895607 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:43.896563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:43.896884 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:43.897161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] PATH_INFO: `/v3/users/02866b31c16f4c799655cf67372fd62f` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:43.914264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] RBAC: Authorizing `identity:get_user(user_id=02866b31c16f4c799655cf67372fd62f)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:43.916353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:43.918477 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 106/217] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:43 2022] GET /identity/v3/users/02866b31c16f4c799655cf67372fd62f => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:43.924161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:44.019080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:44.044531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e576eb-8d0d-422e-9c1d-44e45947b13b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:44.082709 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:44.083824 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.084364 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.084966 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.158376 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:44.159842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:44.244845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:44.262923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 112/218] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:44.268415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:44.385616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab798cd3-38b9-4f76-9a27-3cc345354b3f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:44.387186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:44.433900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb7411-dfa7-4d80-9af3-5a65116fbacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:44.434777 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb7411-dfa7-4d80-9af3-5a65116fbacf None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.435047 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb7411-dfa7-4d80-9af3-5a65116fbacf None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.435315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb7411-dfa7-4d80-9af3-5a65116fbacf None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.436534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb7411-dfa7-4d80-9af3-5a65116fbacf None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:44.438027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb7411-dfa7-4d80-9af3-5a65116fbacf None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:44.440048 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 107/219] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:44 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:44.445805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:44.589494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4956032-022e-4230-9fdb-868d6a145562 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:44.639165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:44.639761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.640054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.640322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.641164 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=02866b31c16f4c799655cf67372fd62f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:44.641499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:44.642993 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05dc62-8da8-4f69-b73e-2c7ebecb40f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:44.652059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 113/220] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:44 2022] GET /identity/v3/role_assignments?user.id=02866b31c16f4c799655cf67372fd62f&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:44.656107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ca98-59ce-47cd-a2a0-9955d9b22e59 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.656386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ca98-59ce-47cd-a2a0-9955d9b22e59 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.656657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ca98-59ce-47cd-a2a0-9955d9b22e59 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.658446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 108/221] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:44.663837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0994-8c8d-4f51-8496-580a04d3f88e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.664131 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0994-8c8d-4f51-8496-580a04d3f88e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.664388 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0994-8c8d-4f51-8496-580a04d3f88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.690529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8b0b0994-8c8d-4f51-8496-580a04d3f88e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:44.691781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8b0b0994-8c8d-4f51-8496-580a04d3f88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:44.757849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:44.791153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 114/222] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:44.825797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ca98-59ce-47cd-a2a0-9955d9b22e59 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:44.930675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181c44b-dc57-4ab4-b73a-ad7059b69f73 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.930979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181c44b-dc57-4ab4-b73a-ad7059b69f73 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.931260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181c44b-dc57-4ab4-b73a-ad7059b69f73 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.932928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 115/223] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:44.938853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.939140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.939415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:44.951767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ca98-59ce-47cd-a2a0-9955d9b22e59 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:44.959094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:44.960777 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:44.993337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:44.994155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:44.994433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:44.994708 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.006105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:45.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:45.015124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:45.020862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 109/224] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:45.026251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.062329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 116/225] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:45.074617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:45.105740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.185284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba5d1-2db0-482d-9b4e-8ce75dd33116 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.247949 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.249000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.249552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.250066 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.251142 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:45.253382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:45.255000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:45.276817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 110/226] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:45.283934 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.289180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77fc9-c756-4e2c-a698-54b78ca73c4f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.309410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:45.329924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.330707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.331048 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.331331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.342495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:45.351528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:45.352408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:45.353209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:45.353842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 117/227] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:45.359910 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.471285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438c2f7-6103-4afa-82bb-dc562c13c693 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.505975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:45.510908 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.513324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.513897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.514522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.516216 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:45.517082 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:45.518867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:45.528225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 118/228] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:45.534270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.535190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d11cd0-e83d-41f4-858e-a8311f8da40c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.567666 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:45.624667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.625598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.625878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.626188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] PATH_INFO: `/v3/users/2ba4c2698c1b472da92acaa8a25e456a` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.627423 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] RBAC: Authorizing `identity:get_user(user_id=2ba4c2698c1b472da92acaa8a25e456a)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:45.628899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:45.637833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 111/229] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/users/2ba4c2698c1b472da92acaa8a25e456a => generated 283 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:45.642311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b903f-4bc0-4a24-a542-52fa214c5cb5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.644014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.682267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.683520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.683987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.684443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] PATH_INFO: `/v3/users/2ba4c2698c1b472da92acaa8a25e456a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.685993 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ba4c2698c1b472da92acaa8a25e456a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:45.687802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:45.690088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 119/230] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/users/2ba4c2698c1b472da92acaa8a25e456a => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:45.695559 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.813461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1acc07-c0ce-42bf-a354-d85c2f51b6d6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.816311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:45.849115 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.850033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.850309 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.850574 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.851750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:45.853355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:45.855569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 120/231] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:45.863028 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:45.904389 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe36c2-66b9-452d-9246-2d5bfe8de40c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.970987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:45.971880 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:45.972160 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:45.972436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:45.973865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:45.975338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:45.977300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 112/232] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:45 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:45.982320 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910f1a0-a1a3-44f7-a511-c5f957e2ef8e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:45.986272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:46.018302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858ff4d-b6dd-47f8-ad94-62c3bb1ade29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:46.019404 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858ff4d-b6dd-47f8-ad94-62c3bb1ade29 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:46.019683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858ff4d-b6dd-47f8-ad94-62c3bb1ade29 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:46.019962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858ff4d-b6dd-47f8-ad94-62c3bb1ade29 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/2ba4c2698c1b472da92acaa8a25e456a/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:46.029378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858ff4d-b6dd-47f8-ad94-62c3bb1ade29 None admin] RBAC: Authorizing `identity:create_grant(project_id=29da51f8b3a24b1485f018c1593a7938, user_id=2ba4c2698c1b472da92acaa8a25e456a, role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:46.029378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858ff4d-b6dd-47f8-ad94-62c3bb1ade29 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:46.043472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 121/233] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:45 2022] PUT /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/2ba4c2698c1b472da92acaa8a25e456a/roles/afd7d26260694a27a2da7aca75966a85 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:20:46.052203 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:46.132499 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d872ba-4e01-4184-81c1-247d6b043e78 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:46.188319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b3e10-ace4-4be8-9a62-50b5ae8064fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:46.189367 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3e10-ace4-4be8-9a62-50b5ae8064fa None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:46.189647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3e10-ace4-4be8-9a62-50b5ae8064fa None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:46.189915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b3e10-ace4-4be8-9a62-50b5ae8064fa None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/2ba4c2698c1b472da92acaa8a25e456a/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:46.194834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b3e10-ace4-4be8-9a62-50b5ae8064fa None admin] RBAC: Authorizing `identity:create_grant(project_id=29da51f8b3a24b1485f018c1593a7938, user_id=2ba4c2698c1b472da92acaa8a25e456a, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:46.196554 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b3e10-ace4-4be8-9a62-50b5ae8064fa None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:46.230058 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 113/234] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:45 2022] PUT /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/2ba4c2698c1b472da92acaa8a25e456a/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:46.930658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b2236-fd13-4600-8b4f-a27a854b18d4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:46.931181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b2236-fd13-4600-8b4f-a27a854b18d4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:46.931181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b2236-fd13-4600-8b4f-a27a854b18d4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:46.932568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 122/235] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:46.940015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:46.940446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:46.940647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:46.963265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:46.964454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:46.988976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bb2a-b2df-48c7-bdbd-1376f99e4c78 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:46.989652 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bb2a-b2df-48c7-bdbd-1376f99e4c78 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:46.990132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7bb2a-b2df-48c7-bdbd-1376f99e4c78 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:46.992160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 123/236] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:46.999188 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e52ef-57f2-4083-a2c5-1592dc350fc7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:46.999842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e52ef-57f2-4083-a2c5-1592dc350fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.000313 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e52ef-57f2-4083-a2c5-1592dc350fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.021623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-310e52ef-57f2-4083-a2c5-1592dc350fc7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:47.023317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-310e52ef-57f2-4083-a2c5-1592dc350fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:47.065692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:47.084839 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 114/237] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:47.108862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.135759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:47.150699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 124/238] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:47.178404 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9202-94a7-456e-a5d3-49b465fc81c1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.178714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9202-94a7-456e-a5d3-49b465fc81c1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.178973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9202-94a7-456e-a5d3-49b465fc81c1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.180372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 125/239] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:47.186219 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.186448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.186703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.206599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:47.208063 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:47.238171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9581ed8-92c0-4f96-be1d-d457a96af550 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:47.281780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:47.283022 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.283371 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.283710 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.293217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 126/240] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:47.294477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.296958 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:47.306742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:47.307646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:47.308489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 115/241] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:47.318334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.396452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:47.414632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de815c53-cf3e-4ea8-84a6-2e49e372335a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:47.423790 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:47.452821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:47.453551 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f46d-476c-4269-8653-d470421d19f8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:47.453845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.454256 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.454626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.456152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:47.457870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:47.464425 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 127/242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:47.465458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.499409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:47.501094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.501586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.502247 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.503437 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:47.503915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:47.506097 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:47.522573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 116/243] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:47.523806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.583533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492098-0351-4f3d-b510-fef4c3427180 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:47.620540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:47.621783 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.622261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.622729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.635707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:47.644180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e586a77-33af-4b0c-8840-69c4524f35f4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:47.647664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:47.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:47.649847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:47.652066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 128/244] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:47.653014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.676000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:47.691306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:47.691992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.692332 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.692609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.693368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:47.694842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:47.704755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:47.725733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 117/245] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:47 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:47.728424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.830494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee21323-8d19-4d97-a3b3-5d598e086d3a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:47.915146 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:47.927866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:47.931162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:47.941176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] PATH_INFO: `/v3/users/015d9a5544b94d93b6ee9f2e88a6e989` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:47.960546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] RBAC: Authorizing `identity:get_user(user_id=015d9a5544b94d93b6ee9f2e88a6e989)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:47.962483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:47.969629 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 129/246] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/users/015d9a5544b94d93b6ee9f2e88a6e989 => generated 297 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:47.970864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:47.979668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827fa49-00e1-429f-a3b7-9f54cc949453 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.050983 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.053126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.053738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.055153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.056135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:48.056776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.058702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.068321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:48.076486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 118/247] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:48.082364 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:48.110698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021d1b-9259-4e87-a7f1-e7f5f839df12 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.149497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.150238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.150574 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.150904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.156296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:48.221765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f5110-8858-4402-84dc-03849feafdd6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.243733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.245221 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.279677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.281198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.281853 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.282717 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] PATH_INFO: `/v3/users/02866b31c16f4c799655cf67372fd62f` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.284362 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] RBAC: Authorizing `identity:get_user(user_id=02866b31c16f4c799655cf67372fd62f)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.286427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.293432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 119/248] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:48 2022] GET /identity/v3/users/02866b31c16f4c799655cf67372fd62f => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:48.300269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:48.350455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:48.385658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 130/249] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:48.391960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:48.423657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bc8f2-cf5c-45a1-bb37-d78cfefd2c32 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.454736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:48.464541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.465693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.465987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.466331 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.467631 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.469916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.472694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 120/250] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:48 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:48.480501 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:48.570750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbb707-50d2-47b6-bca8-dda271af815c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.643960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.644925 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.645203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.645499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.647907 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9ca5-239c-4fed-9786-a429e758f01f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.650270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.651835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.653897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 131/251] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:48 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:48.660119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:48.712549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de1c4-d0a7-4d4e-826b-77ef27fc8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.713480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de1c4-d0a7-4d4e-826b-77ef27fc8883 None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.713760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de1c4-d0a7-4d4e-826b-77ef27fc8883 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.714034 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de1c4-d0a7-4d4e-826b-77ef27fc8883 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/02866b31c16f4c799655cf67372fd62f/roles/9f78fe1c53c24d8ca59e298f2c585211` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.716985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de1c4-d0a7-4d4e-826b-77ef27fc8883 None admin] RBAC: Authorizing `identity:create_grant(project_id=29da51f8b3a24b1485f018c1593a7938, user_id=02866b31c16f4c799655cf67372fd62f, role_id=9f78fe1c53c24d8ca59e298f2c585211)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.718497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de1c4-d0a7-4d4e-826b-77ef27fc8883 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.743873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 121/252] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:48 2022] PUT /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/02866b31c16f4c799655cf67372fd62f/roles/9f78fe1c53c24d8ca59e298f2c585211 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:20:48.745625 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:48.827324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f10e9f-a6a7-4524-9c66-49ab5ea51c65 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:48.871732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:48.872420 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.872699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.872974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.873848 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] RBAC: Adding query filter params (role.id=7366d82e48f645008e175b0e2fe79ce1, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=015d9a5544b94d93b6ee9f2e88a6e989) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:48.874158 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:48.875625 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:48.880255 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37eb73-2a4a-4bae-b678-bf0ee725d9fe None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.880542 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37eb73-2a4a-4bae-b678-bf0ee725d9fe None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.880813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37eb73-2a4a-4bae-b678-bf0ee725d9fe None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.882422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 122/253] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:48.888125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:48.888403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:48.888676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:48.888837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 132/254] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:48 2022] GET /identity/v3/role_assignments?user.id=015d9a5544b94d93b6ee9f2e88a6e989&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=7366d82e48f645008e175b0e2fe79ce1 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:48.937388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:48.938725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:49.020413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:49.046924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 123/255] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:49.079641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:49.208175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde8ac-8075-485f-aa97-c34a56594ca6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:49.217751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:49.252486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf58a92-7c8f-4bb3-8c12-ba48f339ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:49.253342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf58a92-7c8f-4bb3-8c12-ba48f339ae69 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:49.253629 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf58a92-7c8f-4bb3-8c12-ba48f339ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:49.253898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf58a92-7c8f-4bb3-8c12-ba48f339ae69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:49.274238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf58a92-7c8f-4bb3-8c12-ba48f339ae69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5cf58a92-7c8f-4bb3-8c12-ba48f339ae69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:49.283006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:49.283904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:49.284718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 133/256] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:20:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:49.291699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:49.399130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e278e6-38d5-4125-949c-cb619e97c80b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:49.399483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e278e6-38d5-4125-949c-cb619e97c80b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:49.399685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e278e6-38d5-4125-949c-cb619e97c80b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:49.401862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 134/257] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:49.425843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:49.426153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:49.426397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:49.455652 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:49.456882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:49.465287 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba102d6f-cf22-4032-b735-7e6c6daaf392 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:49.574614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 135/258] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:49.601958 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:49.602660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:49.602950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:49.603236 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:49.604004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:49.604312 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:49.605797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:49.609401 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:49.627647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 124/259] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:20:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:49.633443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:49.741771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:49.798415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61805e-bbfe-4048-8717-f265577b51ec None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:49.833865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:49.835501 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:49.835783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:49.836055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] PATH_INFO: `/v3/users/2ba4c2698c1b472da92acaa8a25e456a` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:49.837315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ba4c2698c1b472da92acaa8a25e456a)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:49.838805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:49.840900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 125/260] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:49 2022] GET /identity/v3/users/2ba4c2698c1b472da92acaa8a25e456a => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:49.846703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:49.941580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:49.968030 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec83fab-405a-4e44-8433-1419fbee81f6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:49.971227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884c6f4-4f90-4bc1-bfe9-147e2a697a06 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:49.987521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:50.004993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:50.005682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:50.005961 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:50.006235 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:50.078683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:50.080285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:50.117972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:50.118803 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:50.119085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:50.119359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:50.165899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:50.173755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:50.174669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:50.175582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 136/261] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:20:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 567 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:50.182209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:50.187890 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 126/262] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:50.190209 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:50.207322 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:50.295189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655536b1-61af-47e1-a7ae-490b6f625b97 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:50.330948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:50.331837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:50.332104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:50.332370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:50.333574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:50.335243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:50.341316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 127/263] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:50 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:50.343076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:50.495504 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311af636-bc80-4d7d-b4de-4d449607946e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:50.539430 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:50.582915 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:50.583610 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:50.583896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:50.584175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:50.585014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:50.585387 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:50.586843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:50.587925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcb9dc-a435-46ae-851e-7ec6cb286e62 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:50.607890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 137/264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:50.613450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:50.651721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:50.653685 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:50.654726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:50.655211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:50.655354 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:50.656095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] RBAC: Adding query filter params (role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=2ba4c2698c1b472da92acaa8a25e456a) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:50.656407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:50.662081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:50.669684 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 128/265] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:50 2022] GET /identity/v3/role_assignments?user.id=2ba4c2698c1b472da92acaa8a25e456a&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=afd7d26260694a27a2da7aca75966a85 => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:50.807317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aae95-559b-4772-b013-205d66a15f88 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:50.852227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:50.853127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:50.853434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:50.853703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] PATH_INFO: `/v3/users/2ba4c2698c1b472da92acaa8a25e456a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:50.857956 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ba4c2698c1b472da92acaa8a25e456a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:50.859423 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:50.861521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 138/266] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:50 2022] GET /identity/v3/users/2ba4c2698c1b472da92acaa8a25e456a => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:50.866898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:51.056062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6203c1-98cb-4734-95a4-4df70dc3bbba None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:51.108709 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6800c-428e-4442-9930-de73bca7f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:51.109412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6800c-428e-4442-9930-de73bca7f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.109661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6800c-428e-4442-9930-de73bca7f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.109939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6800c-428e-4442-9930-de73bca7f2d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.132989 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:51.201480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6800c-428e-4442-9930-de73bca7f2d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:51.202972 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6800c-428e-4442-9930-de73bca7f2d6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:51.331843 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 129/267] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:51.337519 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:51.400678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c99116-0aac-47f9-940c-3e5f4570c5a4 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.400885 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c99116-0aac-47f9-940c-3e5f4570c5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.401164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c99116-0aac-47f9-940c-3e5f4570c5a4 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.402586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 130/268] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:51.418939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.419265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.419491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.454372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:51.455605 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:51.468080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:51.488011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d4625-04d4-4173-a086-a2b4abdcc6c1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:51.540352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:51.543260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:51.544172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.544443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.544876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.549301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:51.551387 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:51.553840 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 139/269] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:51 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:51.559478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:51.604113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 131/270] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:51.639074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:51.680711 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544451-5e72-4942-8d7a-be347825160a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:51.725428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:51.726286 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.726565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.727199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.728057 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=2ba4c2698c1b472da92acaa8a25e456a) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:51.728392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:51.730747 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefe0df-bfda-4a4a-beb8-3458c94b8601 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:51.738560 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 140/271] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:51 2022] GET /identity/v3/role_assignments?user.id=2ba4c2698c1b472da92acaa8a25e456a&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:51.774927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9790ee-0077-4768-81f1-d96f1e113014 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.775272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9790ee-0077-4768-81f1-d96f1e113014 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.775472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9790ee-0077-4768-81f1-d96f1e113014 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.777047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 141/272] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:51.783255 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.783557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.783811 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.804174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:51.805415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:51.809718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daabc0b-8945-43fd-b11d-72b9886f4dc5 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:51.842361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:51.886662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 142/273] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:51.902162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9159-c63b-47cd-b924-881bb7aa976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:51.903017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9159-c63b-47cd-b924-881bb7aa976b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.903298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9159-c63b-47cd-b924-881bb7aa976b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.903567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9159-c63b-47cd-b924-881bb7aa976b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.921136 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:51.927620 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9159-c63b-47cd-b924-881bb7aa976b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-5fbc9159-c63b-47cd-b924-881bb7aa976b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:51.944264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:51.945300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:51.946125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 132/274] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:51.947320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa532ac-e98f-4af5-9f1a-d8b48781430e None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.947478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa532ac-e98f-4af5-9f1a-d8b48781430e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.947741 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa532ac-e98f-4af5-9f1a-d8b48781430e None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.949170 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 133/275] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:51.958747 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:51.959061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:51.959313 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:51.978918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:51.980191 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:52.012984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.042418 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde59b2f-646a-4a35-ba3f-611dfd51a65f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.056005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 134/276] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:52.056766 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.078315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3ad9d-32a1-434a-8dc2-975ae72c869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.079145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3ad9d-32a1-434a-8dc2-975ae72c869f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.079423 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3ad9d-32a1-434a-8dc2-975ae72c869f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.079691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3ad9d-32a1-434a-8dc2-975ae72c869f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.080857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3ad9d-32a1-434a-8dc2-975ae72c869f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.082356 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3ad9d-32a1-434a-8dc2-975ae72c869f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.083974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 143/277] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:52.085135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2d8f2-53ab-4e9b-ab9d-9a21851f6310 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.085442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2d8f2-53ab-4e9b-ab9d-9a21851f6310 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.085725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2d8f2-53ab-4e9b-ab9d-9a21851f6310 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.086931 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 144/278] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:52.087627 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2d8f2-53ab-4e9b-ab9d-9a21851f6310 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.105044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.157782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca16402-7d1e-4587-9d31-08204c906727 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.190606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2d8f2-53ab-4e9b-ab9d-9a21851f6310 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.191402 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.192071 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.192353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.192618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.193391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:52.193717 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.195106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.203466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 135/279] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:52.204438 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.225706 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa385-5d93-47e5-a4c5-60eaf9b40d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.226508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa385-5d93-47e5-a4c5-60eaf9b40d65 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.226782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa385-5d93-47e5-a4c5-60eaf9b40d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.227056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa385-5d93-47e5-a4c5-60eaf9b40d65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.234089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.237093 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa385-5d93-47e5-a4c5-60eaf9b40d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c14fa385-5d93-47e5-a4c5-60eaf9b40d65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:52.244778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:52.245746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:52.246555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 145/280] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:52.247226 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.247453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.247730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.264319 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:52.265561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:52.306718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7bd53-9dcf-4607-ab01-af61830218e5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.330483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 146/281] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:52.331294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.338720 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.340117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.340867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.341157 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.341761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.342481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.343936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.364517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 136/282] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:20:52 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:52.365389 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.444412 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c402a-f96f-4da9-b10f-3ccba924d963 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.472328 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.486174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.487090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.487371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.487642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] PATH_INFO: `/v3/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.495282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9be4e-80a9-4a9c-80fe-6eca79f75cdf None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.502716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] RBAC: Authorizing `identity:get_user(user_id=ea2dc5499c7a4718a9ba8b9afb0d5dbd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.504243 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.506104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 147/283] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:52.506923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.542482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.543164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.543444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.543711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.544477 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:52.544780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.546216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.555484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 137/284] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:52.560863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.579284 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.627866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1fcaa-3df7-4a4f-9861-2f35ed91becb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.673469 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.674400 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.674611 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.674842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.682749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4184419-33dd-465b-8e68-de136d1a7304 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.687393 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:52.695452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:52.696367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:52.697196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 148/285] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:52.697902 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.734618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.735602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.735931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.736205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.737062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, user.id=ea2dc5499c7a4718a9ba8b9afb0d5dbd, scope.system=all) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:52.737456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.740069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.746732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 138/286] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/role_assignments?user.id=ea2dc5499c7a4718a9ba8b9afb0d5dbd&scope.system=all&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:52.747658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.764071 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.884057 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc820a-b14f-404d-967a-35e8e7199596 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.910886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6070faf-6113-4a6b-8cd8-b9d9cb1c3b9a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:52.911251 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:52.938488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.939725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.940247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.940713 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] PATH_INFO: `/v3/users/015d9a5544b94d93b6ee9f2e88a6e989` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.942290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] RBAC: Authorizing `identity:get_user(user_id=015d9a5544b94d93b6ee9f2e88a6e989)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.943984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.946141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 149/287] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/users/015d9a5544b94d93b6ee9f2e88a6e989 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:52.952214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:52.983175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:52.983863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:52.984574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:52.984574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:52.985155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:52.985974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:52.987414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:52.996834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 139/288] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:53.002617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:53.077644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd375-ca81-4d66-b575-504c92ad51f5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:53.115489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:53.116789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:53.117297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:53.117939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:53.119634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:53.121465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:53.128341 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 150/289] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:52 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:53.129839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:53.185622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:53.193482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:53.237714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360a867-9893-4a0b-91a1-22d6ad30bcf6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:53.263459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9149819-6640-4097-850f-7bab630aee20 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:53.273969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:53.275235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:53.275708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:53.276170 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/015d9a5544b94d93b6ee9f2e88a6e989/roles/7366d82e48f645008e175b0e2fe79ce1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:53.278445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] RBAC: Authorizing `identity:create_grant(project_id=29da51f8b3a24b1485f018c1593a7938, user_id=015d9a5544b94d93b6ee9f2e88a6e989, role_id=7366d82e48f645008e175b0e2fe79ce1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:53.280460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:53.309412 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 151/290] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:20:53 2022] PUT /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938/users/015d9a5544b94d93b6ee9f2e88a6e989/roles/7366d82e48f645008e175b0e2fe79ce1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:53.328325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:53.329738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:53.330244 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:53.330710 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] PATH_INFO: `/v3/users/02866b31c16f4c799655cf67372fd62f` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:53.332308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] RBAC: Authorizing `identity:get_user(user_id=02866b31c16f4c799655cf67372fd62f)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:53.334108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:53.336032 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 140/291] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:53 2022] GET /identity/v3/users/02866b31c16f4c799655cf67372fd62f => generated 297 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:53.342017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:53.483222 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a9417-3883-4e13-a858-d55a7188fe8a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:53.510193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:53.528422 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:53.529509 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:53.533819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:53.534101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:53.609859 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:53.611745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:53.738046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 152/292] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:53.743701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:53.886916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94b82c-6583-4847-bf6c-a16f39f6fd10 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:53.941267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20217a-819f-4132-b422-4d5477ff784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:53.941267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20217a-819f-4132-b422-4d5477ff784f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:53.941267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20217a-819f-4132-b422-4d5477ff784f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:53.941267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20217a-819f-4132-b422-4d5477ff784f None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:53.941598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20217a-819f-4132-b422-4d5477ff784f None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:53.942291 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20217a-819f-4132-b422-4d5477ff784f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:53.944490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 141/293] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:53 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:53.950335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:53.987128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:54.083722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757949cf-e8d0-43b4-b7a8-c11a5212be31 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:54.126861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:54.127536 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:54.127817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:54.128092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:54.128932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=02866b31c16f4c799655cf67372fd62f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:54.129263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:54.130725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431735-5519-4f1e-b157-f811f898fb57 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:54.137443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 153/294] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:53 2022] GET /identity/v3/role_assignments?user.id=02866b31c16f4c799655cf67372fd62f&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:54.831753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2bbf9-e3ac-464a-a962-fc00d810e575 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:54.832273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2bbf9-e3ac-464a-a962-fc00d810e575 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:54.832273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2bbf9-e3ac-464a-a962-fc00d810e575 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:54.833879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 142/295] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:54.839457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:54.840301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:54.840771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:54.860758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:54.862346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:54.957352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 154/296] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:54.974541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2bbf9-e3ac-464a-a962-fc00d810e575 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:54.998922 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.086973 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2bbf9-e3ac-464a-a962-fc00d810e575 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.121863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03b60b-432d-4953-a23f-b6936d16ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.123088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03b60b-432d-4953-a23f-b6936d16ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.123559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03b60b-432d-4953-a23f-b6936d16ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.124019 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03b60b-432d-4953-a23f-b6936d16ec4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.137017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03b60b-432d-4953-a23f-b6936d16ec4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-7b03b60b-432d-4953-a23f-b6936d16ec4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:55.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:55.146503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:55.147315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:20:55.150288 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 143/297] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:55.156502 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.172292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.266741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6da8db-bb64-4072-b3ac-2c67eb0c93ed None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.302528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.303255 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.303538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.303809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.304560 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:55.304863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:55.306277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:55.312341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb934d1-13d5-45b8-b041-ca1fd09af08a None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.312622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb934d1-13d5-45b8-b041-ca1fd09af08a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.312896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb934d1-13d5-45b8-b041-ca1fd09af08a None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.314567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 144/298] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:55.315413 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 155/299] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:55.319659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d9b1-e99f-429e-9e58-7ccd2222beca None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.319940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d9b1-e99f-429e-9e58-7ccd2222beca None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.320279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d9b1-e99f-429e-9e58-7ccd2222beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.322224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.340395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-88d7d9b1-e99f-429e-9e58-7ccd2222beca None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:55.341935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-88d7d9b1-e99f-429e-9e58-7ccd2222beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:55.406566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.408949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.413413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 145/300] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:55.414918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f41f-1792-4026-bcd3-712fbeadeb87 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.415200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f41f-1792-4026-bcd3-712fbeadeb87 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.415472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f41f-1792-4026-bcd3-712fbeadeb87 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.416850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 146/301] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:55.422379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.422665 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.423011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.440488 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:55.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:55.442897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94916443-4369-45f6-bd2e-58baef969fab None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.476649 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.477565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.477845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.478118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] PATH_INFO: `/v3/users/8e5b00bef2ca47938c0416dfeb014da7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.491735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] RBAC: Authorizing `identity:get_user(user_id=8e5b00bef2ca47938c0416dfeb014da7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:55.493386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:55.495684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 156/302] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/users/8e5b00bef2ca47938c0416dfeb014da7 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:55.496456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.523017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 147/303] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:55.524228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.601196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51198-6f69-484e-824b-9ca390e70e31 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.630927 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d469f4f-7637-48f4-9c1b-0ffab2245efc None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.637227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.638066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.638338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.638603 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.640583 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.650472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:55.658738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:55.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:55.660462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 157/304] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:55.661885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.668869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.679613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.680272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.680550 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.680823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.681678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] RBAC: Adding query filter params (role.id=7366d82e48f645008e175b0e2fe79ce1, user.id=8e5b00bef2ca47938c0416dfeb014da7, scope.system=all) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:55.681988 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:55.683481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:55.690091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 148/305] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/role_assignments?user.id=8e5b00bef2ca47938c0416dfeb014da7&scope.system=all&role.id=7366d82e48f645008e175b0e2fe79ce1 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:55.690848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.793780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90912ff-707c-4aae-acb9-8cb5f6e0e967 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.825910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac4bb9-f502-4c63-b524-98cad1129592 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:55.835994 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.836972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.837405 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.837773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.839480 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:55.841193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:55.848073 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 158/306] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:55.849763 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.867833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:55.869161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.869664 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.870113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.871055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:55.871618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:55.873296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa646fd-8225-41d8-ae8b-c740ff7c3940 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:55.883418 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 149/307] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:55.891870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8794df-99c4-4fad-8134-b07f91ad56a7 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:55.892181 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8794df-99c4-4fad-8134-b07f91ad56a7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:55.892432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8794df-99c4-4fad-8134-b07f91ad56a7 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:55.893989 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 150/308] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:55.894987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8794df-99c4-4fad-8134-b07f91ad56a7 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:55.922019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.964168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:55.967824 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e1f03-93a1-4527-92ab-e2b348184c47 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.004383 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fe425-6026-47cb-86f9-e9f24622fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.005064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fe425-6026-47cb-86f9-e9f24622fdec None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.005384 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fe425-6026-47cb-86f9-e9f24622fdec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.005667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fe425-6026-47cb-86f9-e9f24622fdec None admin] PATH_INFO: `/v3/groups` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.006388 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fe425-6026-47cb-86f9-e9f24622fdec None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:56.007828 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fe425-6026-47cb-86f9-e9f24622fdec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:56.011446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8794df-99c4-4fad-8134-b07f91ad56a7 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.023926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 159/309] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 14:20:55 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:56.023926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.023926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.023926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.044370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:56.045702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:56.063670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.065221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.065660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.066254 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] PATH_INFO: `/v3/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.068114 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] RBAC: Authorizing `identity:get_user(user_id=ea2dc5499c7a4718a9ba8b9afb0d5dbd)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:56.069799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:56.077700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 151/310] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:55 2022] GET /identity/v3/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:56.079859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:56.133878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 160/311] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:56.153427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:56.206108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a9208-3d5b-489f-b549-0c40d64b308e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.226818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:56.235463 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:56.250218 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.251410 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.251858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.252308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] PATH_INFO: `/v3/system/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd/roles/9f78fe1c53c24d8ca59e298f2c585211` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.254275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ea2dc5499c7a4718a9ba8b9afb0d5dbd, role_id=9f78fe1c53c24d8ca59e298f2c585211)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:56.256060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:56.268624 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 152/312] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 14:20:56 2022] PUT /identity/v3/system/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd/roles/9f78fe1c53c24d8ca59e298f2c585211 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:56.281977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f2895-0211-4fb5-814e-e9de41ac736a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.331236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.332051 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.332326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.332590 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.344494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:56.352678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:56.353722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:56.354586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 161/313] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:20:56.361333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:56.473856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84598a-7f3d-45c7-9b76-5e3abd8a7783 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.509639 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.510649 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.511110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.511562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.512485 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:56.512970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:56.514569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:56.523865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 153/314] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:56.529519 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:56.631335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea214761-9828-4909-a181-84c4b1c13f94 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.646551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:56.669836 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.670719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.671002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.671276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] PATH_INFO: `/v3/users/015d9a5544b94d93b6ee9f2e88a6e989` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.672529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] RBAC: Authorizing `identity:get_user(user_id=015d9a5544b94d93b6ee9f2e88a6e989)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:56.674124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:56.676075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 162/315] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:56 2022] GET /identity/v3/users/015d9a5544b94d93b6ee9f2e88a6e989 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:56.682575 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:56.739028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:56.792832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab1329-59eb-4c89-a521-0cc4452e15f6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:56.828668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:56.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:56.829705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:56.829980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:56.900797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:56.902347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:56.964424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:57.024774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 154/316] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:20:57.034317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:57.138621 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f755-e38e-4872-954a-c550dcd0bb66 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:57.173434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bb6d7-cc11-403f-a8c5-7ac4b4429b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:57.174308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bb6d7-cc11-403f-a8c5-7ac4b4429b0c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:57.174588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bb6d7-cc11-403f-a8c5-7ac4b4429b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:57.174867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bb6d7-cc11-403f-a8c5-7ac4b4429b0c None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:57.176064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bb6d7-cc11-403f-a8c5-7ac4b4429b0c None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:57.177536 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bb6d7-cc11-403f-a8c5-7ac4b4429b0c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:57.179416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 163/317] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:20:57 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:57.185459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:57.285948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d4d78-f1ee-40ce-a195-40949a523325 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:57.320105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:57.321082 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:57.321584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:57.322040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:57.323052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] RBAC: Adding query filter params (role.id=7366d82e48f645008e175b0e2fe79ce1, scope.project.id=29da51f8b3a24b1485f018c1593a7938, user.id=015d9a5544b94d93b6ee9f2e88a6e989) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:57.323551 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:57.325205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c2bbe-7915-4371-acb4-b374de4349a7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:57.331927 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 155/318] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:20:57 2022] GET /identity/v3/role_assignments?user.id=015d9a5544b94d93b6ee9f2e88a6e989&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=7366d82e48f645008e175b0e2fe79ce1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:58.068651 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe739a-bae0-4648-8330-40647e6a2afa None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.069193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe739a-bae0-4648-8330-40647e6a2afa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.069193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe739a-bae0-4648-8330-40647e6a2afa None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.070767 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 164/319] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:20:58.075951 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232875db-8bcf-4c31-b46a-af5fac4c6518 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.076369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232875db-8bcf-4c31-b46a-af5fac4c6518 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.076539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232875db-8bcf-4c31-b46a-af5fac4c6518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.093930 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-232875db-8bcf-4c31-b46a-af5fac4c6518 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:58.095201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-232875db-8bcf-4c31-b46a-af5fac4c6518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:58.103452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.165892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 156/320] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:58.188709 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe739a-bae0-4648-8330-40647e6a2afa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.242688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.298561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe739a-bae0-4648-8330-40647e6a2afa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:58.334613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:58.335763 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.336251 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.350198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:58.358687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:58.359744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:20:58.360737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:58.364447 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0808222-5ebd-4f7f-ac06-9fff15a13c60 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.364733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0808222-5ebd-4f7f-ac06-9fff15a13c60 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.365008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0808222-5ebd-4f7f-ac06-9fff15a13c60 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.365477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 165/321] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:58.366724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 157/322] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:58.367024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.373688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.374000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.374274 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.392941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:58.394178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:58.472509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 158/323] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:20:58.473429 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40274924-d408-411c-b052-56d2f5269d2e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:58.485729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.495599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.512956 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:58.513766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.514049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.514431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.515181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:58.515488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:58.516956 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67b353-89a3-4ed8-bac9-d85f6dd3eef0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:58.525444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 166/324] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:58.528667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f9b0e-35cc-44e6-abec-80a12b82a6cf None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.528959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f9b0e-35cc-44e6-abec-80a12b82a6cf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.529636 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f9b0e-35cc-44e6-abec-80a12b82a6cf None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.530906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 167/325] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:20:58.534030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f9b0e-35cc-44e6-abec-80a12b82a6cf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.570013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.603396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da979a-e052-43cc-9c88-a0b369cee0a9 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:58.636199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f9b0e-35cc-44e6-abec-80a12b82a6cf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:58.638869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352c668-0052-46a9-9856-0cf5698a606f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:58.639675 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352c668-0052-46a9-9856-0cf5698a606f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.639954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352c668-0052-46a9-9856-0cf5698a606f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.640224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352c668-0052-46a9-9856-0cf5698a606f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.641412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352c668-0052-46a9-9856-0cf5698a606f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:58.642902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352c668-0052-46a9-9856-0cf5698a606f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:58.644517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 159/326] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:20:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:20:58.645671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.645948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.646216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.663043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:20:58.664250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:20:58.672857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:58.673756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.674044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.674308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] PATH_INFO: `/v3/users/8e5b00bef2ca47938c0416dfeb014da7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.675490 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] RBAC: Authorizing `identity:get_user(user_id=8e5b00bef2ca47938c0416dfeb014da7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:58.676934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:58.678596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 168/327] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:58 2022] GET /identity/v3/users/8e5b00bef2ca47938c0416dfeb014da7 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:58.679357 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.722026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.732638 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 160/328] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:20:58.733394 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.784734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16038c3-e4f1-43b9-a2ba-d984b66f3f21 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:58.799647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.819893 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:58.820568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.820848 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.821140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] PATH_INFO: `/v3/groups` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.821858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:58.823316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:58.830414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8166a-c59a-46bb-893a-38aecd116ff8 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:58.836819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 169/329] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 14:20:58 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:20:58.838013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:58.880676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:58.881638 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:58.881919 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:58.882193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] PATH_INFO: `/v3/system/users/8e5b00bef2ca47938c0416dfeb014da7/roles/7366d82e48f645008e175b0e2fe79ce1` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:58.883823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8e5b00bef2ca47938c0416dfeb014da7, role_id=7366d82e48f645008e175b0e2fe79ce1)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:58.886196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:58.896978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 161/330] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 14:20:58 2022] PUT /identity/v3/system/users/8e5b00bef2ca47938c0416dfeb014da7/roles/7366d82e48f645008e175b0e2fe79ce1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:20:58.961276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:58.966150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd430b-c322-486f-9f6d-4ea6dddfb425 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:59.009893 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:59.010776 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:59.011163 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:59.011442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:59.024837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:20:59.033496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:20:59.034540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:20:59.035443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 170/331] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:20:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:20:59.040933 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:59.157643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be2143-3239-4b23-b6fb-c9a1d70c90cd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:59.194289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:59.194981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:59.195249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:59.195519 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:59.196276 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:59.196654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:59.198072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:59.199689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:20:59.209309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 162/332] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:20:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:59.213362 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:59.321345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd78c91-b39c-410e-a4be-963471fba1f7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:59.355898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f0dcd-38b6-450d-8b63-695997370376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:59.357064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f0dcd-38b6-450d-8b63-695997370376 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:59.357415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f0dcd-38b6-450d-8b63-695997370376 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:59.357693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f0dcd-38b6-450d-8b63-695997370376 None admin] PATH_INFO: `/v3/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:59.358895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f0dcd-38b6-450d-8b63-695997370376 None admin] RBAC: Authorizing `identity:get_user(user_id=ea2dc5499c7a4718a9ba8b9afb0d5dbd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:59.360420 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f0dcd-38b6-450d-8b63-695997370376 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:59.362337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 171/333] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:20:59 2022] GET /identity/v3/users/ea2dc5499c7a4718a9ba8b9afb0d5dbd => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:20:59.369331 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:20:59.479105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81fff4-2a66-49b3-83fe-7de3e62c7936 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:20:59.513985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:20:59.515007 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:20:59.515523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:20:59.515908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:20:59.516908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] RBAC: Adding query filter params (role.id=9f78fe1c53c24d8ca59e298f2c585211, user.id=ea2dc5499c7a4718a9ba8b9afb0d5dbd, scope.system=all) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:20:59.517374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:20:59.518920 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f553758-dff6-4209-9541-bccf6df3166e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:20:59.537391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 163/334] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:20:59 2022] GET /identity/v3/role_assignments?user.id=ea2dc5499c7a4718a9ba8b9afb0d5dbd&scope.system=all&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:01.315098 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534ef0-9f1a-463b-be12-1920b4c2a3e1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:01.315616 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534ef0-9f1a-463b-be12-1920b4c2a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:01.315616 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534ef0-9f1a-463b-be12-1920b4c2a3e1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:01.317080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 172/335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:01.322868 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fd339-a9d6-4d1c-9889-beffeef497c5 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:01.323751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fd339-a9d6-4d1c-9889-beffeef497c5 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:01.324223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fd339-a9d6-4d1c-9889-beffeef497c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:01.343009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-879fd339-a9d6-4d1c-9889-beffeef497c5 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:01.344611 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-879fd339-a9d6-4d1c-9889-beffeef497c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:01.409998 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:01.433122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 164/336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:01.451488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec9d9-2288-4c2d-8ad9-5f67258f7f55 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:01.451789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec9d9-2288-4c2d-8ad9-5f67258f7f55 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:01.452044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec9d9-2288-4c2d-8ad9-5f67258f7f55 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:01.453464 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 173/337] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:01.461400 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:01.462120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:01.462597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:01.476572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec9d9-2288-4c2d-8ad9-5f67258f7f55 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:01.493414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:01.495034 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:01.537970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:01.631609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 165/338] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:01.658435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec9d9-2288-4c2d-8ad9-5f67258f7f55 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:01.678738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:01.718390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:01.719307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:01.719665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:01.719940 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:01.737833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:01.746290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:01.747190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:01.748411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 174/339] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:01.766665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:01.802290 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8dd94-fc04-4387-aea5-349db9ed29f1 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:01.818675 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:01.854089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:01.854089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:01.854089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:01.854089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:01.871439 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:01.887699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:01.888627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:01.889480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 166/340] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:01.897078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:01.930733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:01.961197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae7300-3d6a-4fb0-9b78-57c9a7346e45 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:02.028026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:02.028744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.029391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.029681 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.030436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:02.030735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:02.032155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315d76d-9f8c-4170-96c5-0eb6a7ffc0d4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:02.048567 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 175/341] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:02.051193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60039ce-c8fd-4473-b177-b93dc66d3509 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.051472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60039ce-c8fd-4473-b177-b93dc66d3509 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.051742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60039ce-c8fd-4473-b177-b93dc66d3509 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.052987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 176/342] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:02.059112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60039ce-c8fd-4473-b177-b93dc66d3509 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:02.086307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df153ad-af5c-4518-86bc-27bd07c226bc None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:02.156294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:02.156909 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.157247 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.157533 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.158301 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:02.158611 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:02.160023 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302fbc6-1747-4cf7-90f8-1da6f14ad994 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:02.175448 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 167/343] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:02.175448 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.175448 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.175448 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.196894 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:02.198189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:02.199015 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60039ce-c8fd-4473-b177-b93dc66d3509 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:02.250566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:02.252814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:02.254077 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.254358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.254627 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] PATH_INFO: `/v3/users/8e5b00bef2ca47938c0416dfeb014da7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.256709 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] RBAC: Authorizing `identity:get_user(user_id=8e5b00bef2ca47938c0416dfeb014da7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:02.258851 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:02.264263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 177/344] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:21:02 2022] GET /identity/v3/users/8e5b00bef2ca47938c0416dfeb014da7 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:02.265052 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:02.307338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:02.327893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 168/345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:02.329959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:02.453580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aa24d-8ec1-4d07-a39e-638e497a34d0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:02.502994 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10bf4b-f7a0-41e9-a9ef-3067786f4bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:02.503672 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10bf4b-f7a0-41e9-a9ef-3067786f4bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.503953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10bf4b-f7a0-41e9-a9ef-3067786f4bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.504224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10bf4b-f7a0-41e9-a9ef-3067786f4bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.527395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8518f10-9944-4b4c-8dd8-3df4277da2df None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:02.606816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:02.607507 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.607780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.608057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.608896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] RBAC: Adding query filter params (role.id=7366d82e48f645008e175b0e2fe79ce1, user.id=8e5b00bef2ca47938c0416dfeb014da7, scope.system=all) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:02.609197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:02.610736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:02.615758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10bf4b-f7a0-41e9-a9ef-3067786f4bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:02.617212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10bf4b-f7a0-41e9-a9ef-3067786f4bc1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:02.637511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:02.650707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 169/346] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:21:02 2022] GET /identity/v3/role_assignments?user.id=8e5b00bef2ca47938c0416dfeb014da7&scope.system=all&role.id=7366d82e48f645008e175b0e2fe79ce1 => generated 510 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:02.651614 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:02.729756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:02.824526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 178/347] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:02.825956 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404323a-0c6b-40e2-9bba-5236a09296e6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.826242 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404323a-0c6b-40e2-9bba-5236a09296e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.826517 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404323a-0c6b-40e2-9bba-5236a09296e6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.828141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 179/348] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:02.834584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.834874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.835137 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.865340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:02.866569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:02.872223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50372-eee4-4bb2-ba66-c95aaf69dbec None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:02.918303 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:02.919163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:02.919444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:02.919722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:02.945266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:02.957688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:02.958798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:02.959632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 170/349] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:02.965050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.013359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 180/350] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:03.013359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.093276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.172323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcbe2a-779b-48b6-96a5-016c3db1c9e7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.174368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.200694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7558-e8ac-4ccd-8161-de3ee3a28e23 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.217370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.218066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.218378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.218720 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.219658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:03.219972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:03.221410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:03.231808 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 181/351] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:03.232662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.252016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.254615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.254893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.255161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.256648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:03.258816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:03.260719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 171/352] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:02 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:03.262592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.361491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a455abe-bc10-4e5a-a9d3-712c6ecda278 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.371786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.373065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e150a-6612-4231-a10d-211a47c0a9d9 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.402186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.403062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.403342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.403615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.414406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.415061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.415336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.415605 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.417286 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:03.426017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:03.426968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:03.427802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 182/353] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:03.428995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.439526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.498040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:03.499750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:03.555954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c49d0-1a39-46ef-8fd0-06c31521a48d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.598019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.599053 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.599338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.599660 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.606780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:03.608352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:03.611099 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 183/354] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:03.611953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.612964 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 172/355] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:03.622280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.653540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.690608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.743301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20941430-932f-4460-b807-37e30ea2e052 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.750199 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9c176-c1f4-4460-9ca5-f8593824baa1 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.789914 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.790353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:03.790701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.791334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.791435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.791520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.791591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.791591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.792068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=730c37dd34844384872ab44ea74c7db5) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:03.792304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:03.792406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:03.792614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:03.796787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:03.796928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8779-a35a-4918-885d-c69a52ee16c3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:03.803334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 173/356] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:03.804197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.809739 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 184/357] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:03.811290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b24fd5-573d-488d-8b4e-748c89cd21f6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:03.811571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b24fd5-573d-488d-8b4e-748c89cd21f6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:03.811849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b24fd5-573d-488d-8b4e-748c89cd21f6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:03.814396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 185/358] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:03.815216 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b24fd5-573d-488d-8b4e-748c89cd21f6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:03.949985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:03.960439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b24fd5-573d-488d-8b4e-748c89cd21f6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.964741 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dead8-0ec6-42bf-8d6d-c0b0b0b0ff95 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:03.984413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.000172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a778445-8d93-47da-957a-f0d21a92fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.001673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a778445-8d93-47da-957a-f0d21a92fa1d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.002185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a778445-8d93-47da-957a-f0d21a92fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.002703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a778445-8d93-47da-957a-f0d21a92fa1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.013904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d63a9-73b9-4005-84ee-f300ce8aeb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.014863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d63a9-73b9-4005-84ee-f300ce8aeb4d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.015137 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d63a9-73b9-4005-84ee-f300ce8aeb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.015406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d63a9-73b9-4005-84ee-f300ce8aeb4d None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.016781 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d63a9-73b9-4005-84ee-f300ce8aeb4d None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.018534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d63a9-73b9-4005-84ee-f300ce8aeb4d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.021555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 174/359] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:04.027868 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.028230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.028506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.049739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:04.050959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:04.095904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a778445-8d93-47da-957a-f0d21a92fa1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.097941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a778445-8d93-47da-957a-f0d21a92fa1d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.138378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 175/360] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:04.141163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.226312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.237745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 186/361] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:04.238836 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54587-9dcf-4351-8bc4-ca68165bcce4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.239125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54587-9dcf-4351-8bc4-ca68165bcce4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.239399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54587-9dcf-4351-8bc4-ca68165bcce4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.242362 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 187/362] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:04.243169 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54587-9dcf-4351-8bc4-ca68165bcce4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.280369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b0a5a-2367-4c6b-8c26-9c6c314f8095 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.298822 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.325698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.326738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.327019 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.327293 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.329306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.330308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.332334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 176/363] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:04.333887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.371896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54587-9dcf-4351-8bc4-ca68165bcce4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.414180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d932c95-e364-4b51-84e7-310dc22faac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.415014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932c95-e364-4b51-84e7-310dc22faac5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.416536 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932c95-e364-4b51-84e7-310dc22faac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.416536 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932c95-e364-4b51-84e7-310dc22faac5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.429678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d932c95-e364-4b51-84e7-310dc22faac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0d932c95-e364-4b51-84e7-310dc22faac5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:04.438602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:04.439523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:04.440354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:04.443093 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 188/364] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:04.444609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.445068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.445578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.463815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:04.465417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:04.467071 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821195e-46ef-498b-b9be-e67c8261b7e3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.517139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.518382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.518859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.519343 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.520756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.522454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.525139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 177/365] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:04.526184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.531130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.558019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 189/366] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:04.558761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.584767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.655162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f867f-bcd2-421d-81f5-d79f756542f9 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.690430 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e7293-668e-452e-8c5d-792a6323ebc3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.693492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.694171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.694443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.694718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.695559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=730c37dd34844384872ab44ea74c7db5) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:04.695867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.699384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.706227 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 178/367] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=afd7d26260694a27a2da7aca75966a85 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:04.706569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.735671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.736369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.736644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.736907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.738148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:04.739345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.739837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.755766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 190/368] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:04.757367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.820414 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.847734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588970b-10f6-413f-a560-3dae095ba3c7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.863776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:04.882876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4c10-0ba8-4aa8-b369-6166e3fa7266 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:04.891997 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.892696 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.893084 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.893489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.895030 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:04.896658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:04.898866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 179/369] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:04.899892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:04.922739 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:04.923943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:04.924396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:04.925650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:04.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:04.946043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:04.946969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 191/370] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:04.948322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:05.016834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e74d2-ed3e-40b5-ae47-2f99e17f69b0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.064116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.064788 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.065069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.065407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.071922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76e002-0349-41e8-8e26-32db6f60bbe7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.082540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:05.123554 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.124335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.124685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.126899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.127842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=29da51f8b3a24b1485f018c1593a7938) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:05.128228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.129921 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.133918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:05.136373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 192/371] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:05.137377 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:05.163337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.165848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.270951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef16ac7-84c9-46f9-8881-f772b383b8c6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.310347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 180/372] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:05.315677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:05.316168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.317000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.317379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.317752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.318614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:05.319162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.320498 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.330209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 193/373] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:05.336030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:05.407228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:05.465612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:05.488031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6623c-4f6a-4669-95de-e7b53e454ab3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.530296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.532541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.532835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.533113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.535690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.537188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.543148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed8a65-93fb-4ddb-98a5-db53726cc56a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.544037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 181/374] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:05 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:05.544899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:05.599214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.599871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.600077 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.605259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.739004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690033eb-f33b-4adb-b782-bb558d006aa2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.782448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.783933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.789994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.790881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.791162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.791434 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.792860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.794376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.796373 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 182/375] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:05 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:05.802718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:05.827965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:05.878618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:05.926978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdba46c-69f5-4ec0-89f9-7545a88c28e7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:05.970177 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:05.970850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:05.971132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:05.971401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:05.972247 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=29da51f8b3a24b1485f018c1593a7938) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:05.972559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:05.974042 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:05.995595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 183/376] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:05 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=afd7d26260694a27a2da7aca75966a85 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:06.029512 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 194/377] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:06.035021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:06.141338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:06.168021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c7865-6d35-426b-abff-927ca3d90a4b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:06.222462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c98d4-b466-4d3e-a005-65fa88ad9f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:06.223734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c98d4-b466-4d3e-a005-65fa88ad9f57 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.224214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c98d4-b466-4d3e-a005-65fa88ad9f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.224701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c98d4-b466-4d3e-a005-65fa88ad9f57 None admin] PATH_INFO: `/v3/projects/b58c8de9649042479ca5651859a02fd6` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.230088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c98d4-b466-4d3e-a005-65fa88ad9f57 None admin] RBAC: Authorizing `identity:get_project(project_id=b58c8de9649042479ca5651859a02fd6)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:06.231924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c98d4-b466-4d3e-a005-65fa88ad9f57 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:06.234408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 184/378] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:06 2022] GET /identity/v3/projects/b58c8de9649042479ca5651859a02fd6 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:06.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:06.352392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd483f2-c3a5-4700-8050-e3cef2356041 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.353019 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd483f2-c3a5-4700-8050-e3cef2356041 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.353560 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd483f2-c3a5-4700-8050-e3cef2356041 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.355384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 185/379] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:21:06.361469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.361763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.362045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.374342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac1acb-5880-449b-b74e-c569cf298475 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:06.393132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:06.396341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:06.424413 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:06.429749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:06.430879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.431346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.431798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] PATH_INFO: `/v3/groups/b635485920c1435e8f03f97b11ca442d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.443059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] RBAC: Authorizing `identity:get_group(group_id=b635485920c1435e8f03f97b11ca442d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:06.444799 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:06.448128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 195/380] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:06 2022] GET /identity/v3/groups/b635485920c1435e8f03f97b11ca442d => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:06.452554 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:06.536471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 186/381] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:06.567076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:06.596846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8939db-6e79-40ab-8dce-7b0bb11c7e3e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:06.646186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:06.646806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.647049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.647302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.648149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] RBAC: Adding query filter params (group.id=b635485920c1435e8f03f97b11ca442d, role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=b58c8de9649042479ca5651859a02fd6) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:06.648464 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:06.650381 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:06.662119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 196/382] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:06 2022] GET /identity/v3/role_assignments?group.id=b635485920c1435e8f03f97b11ca442d&scope.project.id=b58c8de9649042479ca5651859a02fd6&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:06.667234 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:06.689641 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183bdab-ef61-411e-b460-91e8943a220e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:06.731546 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:06.732804 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.733303 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.733831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.745533 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:06.754026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:06.754982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:06.755827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:06.758656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 187/383] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:06.764163 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:06.837044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:06.887506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f941607-a44a-4709-8fc5-cd5f0e3dd95e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:06.929359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:06.930406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:06.930882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:06.931358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:06.932403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:06.932903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:06.934592 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:06.945390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 197/384] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:06.951319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:07.101471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b60e3-d129-4bee-b4fd-f598f8b0595a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:07.145336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:07.146215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:07.146493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:07.146765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:07.148832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:07.150366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:07.152384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 188/385] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:06 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:07.158007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:07.268985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3cc8d-3c55-4543-bc10-04d559e0941a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:07.292619 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:07.304977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9064-dc3c-4728-a81e-0e540ea0a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:07.306147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9064-dc3c-4728-a81e-0e540ea0a54c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:07.306489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9064-dc3c-4728-a81e-0e540ea0a54c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:07.306970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9064-dc3c-4728-a81e-0e540ea0a54c None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:07.308165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9064-dc3c-4728-a81e-0e540ea0a54c None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:07.309640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9064-dc3c-4728-a81e-0e540ea0a54c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:07.311713 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 198/386] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:07 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:07.317188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:07.337565 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:07.440052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efd44-bb8c-4807-a10b-cc59b227bc72 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:07.479917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b956f-effe-486b-96da-bfba48c8ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:07.481129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b956f-effe-486b-96da-bfba48c8ddfb None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:07.481601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b956f-effe-486b-96da-bfba48c8ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:07.482045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b956f-effe-486b-96da-bfba48c8ddfb None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5/groups/fdff84090dc546c4919120f59db76122/roles/9f78fe1c53c24d8ca59e298f2c585211` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:07.485996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b956f-effe-486b-96da-bfba48c8ddfb None admin] RBAC: Authorizing `identity:create_grant(project_id=730c37dd34844384872ab44ea74c7db5, group_id=fdff84090dc546c4919120f59db76122, role_id=9f78fe1c53c24d8ca59e298f2c585211)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:07.487738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b956f-effe-486b-96da-bfba48c8ddfb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:07.524603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 189/387] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:21:07 2022] PUT /identity/v3/projects/730c37dd34844384872ab44ea74c7db5/groups/fdff84090dc546c4919120f59db76122/roles/9f78fe1c53c24d8ca59e298f2c585211 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:21:08.045962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034df5a3-67f3-4e94-9f2f-74220a73c79d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.046511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034df5a3-67f3-4e94-9f2f-74220a73c79d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.046511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034df5a3-67f3-4e94-9f2f-74220a73c79d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.048194 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 199/388] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:08.053750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b82f-8a7d-416a-91f9-7d26a0e86337 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.054177 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b82f-8a7d-416a-91f9-7d26a0e86337 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.054353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b82f-8a7d-416a-91f9-7d26a0e86337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.078284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-a412b82f-8a7d-416a-91f9-7d26a0e86337 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:08.079576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-a412b82f-8a7d-416a-91f9-7d26a0e86337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:08.153318 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:08.163159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 190/389] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:08.186243 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57d1a3-882f-4adf-8ecb-c5f710dd35ce None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.186431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57d1a3-882f-4adf-8ecb-c5f710dd35ce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.186694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57d1a3-882f-4adf-8ecb-c5f710dd35ce None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.188233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 200/390] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:08.193835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.194384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.194874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.197581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57d1a3-882f-4adf-8ecb-c5f710dd35ce None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:08.219313 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:08.220561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:08.327185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57d1a3-882f-4adf-8ecb-c5f710dd35ce None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:08.368843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:08.370076 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.370571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.371037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.383825 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:08.398152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:08.399090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:08.399947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:08.409268 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 191/392] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:08.409764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 201/392] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:08.413450 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:08.435330 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:08.506255 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:08.599907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf631e-1bf7-4268-bc39-12478c76fe9f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:08.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f7a87-b615-4e49-b708-b2f0ef7e1263 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:08.654369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:08.655213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.655493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.655762 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.669626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:08.681040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:08.685939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:08.686636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.686917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.687182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.687948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:08.688301 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:08.689755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:08.690642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:08.691516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:08.692328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:08.692941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 202/393] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:08.702305 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:08.704490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 192/394] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:08.710283 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:08.820364 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b923393-f070-42d4-88e3-18541c7e0d8a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:08.863702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9992fb-e62a-4ec2-95bf-02098c7e36e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:08.865260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9992fb-e62a-4ec2-95bf-02098c7e36e2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.865954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9992fb-e62a-4ec2-95bf-02098c7e36e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.866423 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9992fb-e62a-4ec2-95bf-02098c7e36e2 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.868622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9992fb-e62a-4ec2-95bf-02098c7e36e2 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:08.870329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9992fb-e62a-4ec2-95bf-02098c7e36e2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:08.872830 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 193/395] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:08 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:08.874912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62caac-a8f0-4004-a337-d2488b064ab4 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.875704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62caac-a8f0-4004-a337-d2488b064ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.876186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62caac-a8f0-4004-a337-d2488b064ab4 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.883975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 194/396] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:08.884916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62caac-a8f0-4004-a337-d2488b064ab4 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:08.914110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461b5e-e7d8-4bdb-9843-224a1f226235 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:08.928883 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:08.958922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:08.960026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.960367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.960644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.961766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:08.962083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:08.963787 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36632b08-1e76-47b9-a885-8436e6374f4b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:08.974434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 203/397] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:08.974713 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:08.975786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:08.976082 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:08.976363 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:08.997407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:08.998691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:09.031666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62caac-a8f0-4004-a337-d2488b064ab4 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.073648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.074613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.074990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.075359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.076676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.078429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.083820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 195/398] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:08 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:09.083820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.084603 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 204/399] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:09.090739 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.190202 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283b06-3f6e-4ab9-ab9d-0b4d30572693 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.216084 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3a70-d41a-45ba-8bd9-99cff7dcc0b2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.217496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:09.219267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:09.229456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.230514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.230886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.231344 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.232587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.234289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.236564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 196/400] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:09.237679 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.260223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.261458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.262040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.262453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5/groups/fdff84090dc546c4919120f59db76122/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.265161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] RBAC: Authorizing `identity:create_grant(project_id=730c37dd34844384872ab44ea74c7db5, group_id=fdff84090dc546c4919120f59db76122, role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.267127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.286127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 205/401] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:21:09 2022] PUT /identity/v3/projects/730c37dd34844384872ab44ea74c7db5/groups/fdff84090dc546c4919120f59db76122/roles/afd7d26260694a27a2da7aca75966a85 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:21:09.294700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.347261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb1dfa-ba5b-4183-a6af-60bec910f456 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.388228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.393907 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.394278 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.394517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.419107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:09.427776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:09.428762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:09.429648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 197/402] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:09.438560 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.477316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a5dac-c95d-4f0e-8257-f485557b8eae None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.500427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:09.522648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19931bbe-ddd1-41db-ad34-9da598c21e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.523534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19931bbe-ddd1-41db-ad34-9da598c21e00 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.523815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19931bbe-ddd1-41db-ad34-9da598c21e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.524088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19931bbe-ddd1-41db-ad34-9da598c21e00 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.525367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19931bbe-ddd1-41db-ad34-9da598c21e00 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.527462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19931bbe-ddd1-41db-ad34-9da598c21e00 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.529953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 206/403] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:09.532954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2342d-f8a3-4cf3-abcd-83355fff68b1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.533465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2342d-f8a3-4cf3-abcd-83355fff68b1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.533876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2342d-f8a3-4cf3-abcd-83355fff68b1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.540402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 207/404] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:09.540402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2342d-f8a3-4cf3-abcd-83355fff68b1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.569125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163452-c974-4a9b-beec-bd34d6753535 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.572978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:09.621290 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.621984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.622261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.622535 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.623298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:09.623604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.625694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8752e00-3f97-464f-8908-75c5eaa68844 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.641369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 198/405] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:09.644094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.644386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.644798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.659856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2342d-f8a3-4cf3-abcd-83355fff68b1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.663192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:09.663966 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:09.702500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.703883 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.704402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.704866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938/groups/fdff84090dc546c4919120f59db76122/roles/9f78fe1c53c24d8ca59e298f2c585211` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.707561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] RBAC: Authorizing `identity:create_grant(project_id=29da51f8b3a24b1485f018c1593a7938, group_id=fdff84090dc546c4919120f59db76122, role_id=9f78fe1c53c24d8ca59e298f2c585211)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.709731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.729071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 208/406] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:21:09 2022] PUT /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938/groups/fdff84090dc546c4919120f59db76122/roles/9f78fe1c53c24d8ca59e298f2c585211 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:09.730370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.747447 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 199/407] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:09.782835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:09.841443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:09.852952 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a3972-2778-4c41-b933-2beef308d65f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:09.896741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:09.897598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:09.898114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:09.898533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:09.898899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:09.900395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:09.901993 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:09.904835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 209/408] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:09.910870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.026345 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaab04-8e9d-41e1-af51-3708852cab4a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.029569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666033e0-bfe6-41ca-8113-7a38d5889ebe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.076985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.077916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.078188 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.078460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.079704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:10.081148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:10.083283 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 210/409] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:10.089984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.090807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.091086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.091355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.094125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.103242 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:10.114647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:10.115578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:10.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 200/410] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:10.121963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.149800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:10.261490 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d52d4-9571-465e-a155-4701d69a5a7a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.304567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab551977-7cca-4b77-b80f-0e09c59c5c63 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.305710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7d43a-a58e-4181-b8c3-67b316bf1984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.306889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7d43a-a58e-4181-b8c3-67b316bf1984 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.307363 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7d43a-a58e-4181-b8c3-67b316bf1984 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.307828 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7d43a-a58e-4181-b8c3-67b316bf1984 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938/groups/fdff84090dc546c4919120f59db76122/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.314135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:10.315809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7d43a-a58e-4181-b8c3-67b316bf1984 None admin] RBAC: Authorizing `identity:create_grant(project_id=29da51f8b3a24b1485f018c1593a7938, group_id=fdff84090dc546c4919120f59db76122, role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:10.317545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7d43a-a58e-4181-b8c3-67b316bf1984 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:10.348445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.349382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.349765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.350173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.351079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:10.351538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:10.352675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 211/411] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:21:10 2022] PUT /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938/groups/fdff84090dc546c4919120f59db76122/roles/afd7d26260694a27a2da7aca75966a85 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:10.353202 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:10.354455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f534175-6388-42c6-b61a-0969729bf168 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.355142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f534175-6388-42c6-b61a-0969729bf168 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.355565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f534175-6388-42c6-b61a-0969729bf168 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.362694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 212/412] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:10.365223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.365743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.366213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.366629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 201/413] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:10.371978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.405537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:10.407981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:10.512436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8e7bd-d0b4-451d-8185-0e36446bdc4d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.548278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 213/414] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:10.553819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.554936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.555418 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.555903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] PATH_INFO: `/v3/groups/b635485920c1435e8f03f97b11ca442d` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.558080 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] RBAC: Authorizing `identity:get_group(group_id=b635485920c1435e8f03f97b11ca442d)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:10.559746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:10.561893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 202/415] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:10 2022] GET /identity/v3/groups/b635485920c1435e8f03f97b11ca442d => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:10.567425 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.573146 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.594457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:10.613992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:10.713366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3665a5-fd65-4926-862b-d582112b18d9 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.778881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.780872 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.781156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.781470 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.783927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccc1ba-d8eb-45b3-8238-2009f0838ab8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:10.805533 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:10.816848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:10.817820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:10.818630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:10.829633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 203/416] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:10.833726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.855238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:10.857432 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:10.857719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:10.857991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] PATH_INFO: `/v3/projects/b58c8de9649042479ca5651859a02fd6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:10.861124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] RBAC: Authorizing `identity:get_project(project_id=b58c8de9649042479ca5651859a02fd6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:10.862618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:10.869612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 214/417] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:10 2022] GET /identity/v3/projects/b58c8de9649042479ca5651859a02fd6 => generated 344 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:10.875139 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:10.966095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:10.972450 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467b33-d931-44d1-837d-1598062d7558 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:11.012328 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:11.013010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:11.013340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:11.013576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:11.014320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:11.014623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:11.016007 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:11.029401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 204/418] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:11.031196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:11.080765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:11.114939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c3375-c242-489b-84fa-162672c123f8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:11.187394 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12565c-e3dd-49e6-857e-35df1b299a93 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:11.193431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:11.194805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:11.195297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:11.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] PATH_INFO: `/v3/projects/b58c8de9649042479ca5651859a02fd6/groups/b635485920c1435e8f03f97b11ca442d/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:11.202729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] RBAC: Authorizing `identity:create_grant(project_id=b58c8de9649042479ca5651859a02fd6, group_id=b635485920c1435e8f03f97b11ca442d, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:11.204693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:11.229694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 215/419] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:21:10 2022] PUT /identity/v3/projects/b58c8de9649042479ca5651859a02fd6/groups/b635485920c1435e8f03f97b11ca442d/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:11.245214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:11.246321 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:11.246802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:11.247336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:11.285470 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:11.341016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:11.342550 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:11.474606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 205/420] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:11.480146 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:11.592604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d722a-23bf-4673-a7e9-946a914edf1d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:11.629604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:11.630489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:11.630770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:11.631044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:11.632245 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:11.633733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:11.635764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 216/421] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:11 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:11.641078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:11.710333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:11.759230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8edb186-c8bf-4974-99e8-a88012666838 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:11.799922 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40ae62-f188-496a-a7ae-c7451fd16c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:11.800794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40ae62-f188-496a-a7ae-c7451fd16c8e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:11.801073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40ae62-f188-496a-a7ae-c7451fd16c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:11.801373 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40ae62-f188-496a-a7ae-c7451fd16c8e None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:11.805979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40ae62-f188-496a-a7ae-c7451fd16c8e None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:11.807460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40ae62-f188-496a-a7ae-c7451fd16c8e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:11.809141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 206/422] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:11 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:11.815105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:11.830774 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:11.940014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f8686-ca66-4f17-9184-bdafa6970da4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:11.988302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:11.988972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:11.989268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:11.989543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:11.990387 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=730c37dd34844384872ab44ea74c7db5) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:11.990694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:11.992132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4a9df-fd65-4b4a-8fbe-efb07a301e0e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:11.998633 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 217/423] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:11 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:12.018143 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a481f-d2fc-4fd1-9e37-777c616067a7 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.018320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a481f-d2fc-4fd1-9e37-777c616067a7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.018602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a481f-d2fc-4fd1-9e37-777c616067a7 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.020017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 207/424] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:12.027185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155fcb6-58b9-4dd8-bccf-d608f70bf414 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.027529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155fcb6-58b9-4dd8-bccf-d608f70bf414 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.027741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155fcb6-58b9-4dd8-bccf-d608f70bf414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.053210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9155fcb6-58b9-4dd8-bccf-d608f70bf414 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:12.054428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9155fcb6-58b9-4dd8-bccf-d608f70bf414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:12.166860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 218/425] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:12.202894 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a481f-d2fc-4fd1-9e37-777c616067a7 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:12.317745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a481f-d2fc-4fd1-9e37-777c616067a7 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:12.347380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:12.361560 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:12.362399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.362675 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.362944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.377401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:12.382208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b8988-8782-41de-8c26-a9bc569c309a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.382722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b8988-8782-41de-8c26-a9bc569c309a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.383220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b8988-8782-41de-8c26-a9bc569c309a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.385147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 219/426] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:12.388908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:12.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:12.390792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 208/427] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:12.391552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.391552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.391552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.393054 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:12.406900 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:12.412472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:12.413975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:12.493535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 220/428] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:12.516701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7227ee-2cc5-482d-a823-ea67c3ff2d90 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:12.519924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:12.565608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:12.566656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.567151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.567633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.573626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:12.574272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:12.575925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:12.585678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 209/429] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:12.591093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:12.661346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdbf51-cd09-436b-b2c1-b1cd6caddcb3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:12.666574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:12.710752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:12.711713 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.713625 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.713904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.719565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:12.729110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:12.736152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14de57-68a5-4f95-8ef4-775b1d700764 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:12.737765 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:12.738829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:12.739839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 221/430] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:12.754247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:12.786041 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e2391-fe54-4636-bc4d-99fffe84479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:12.786751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e2391-fe54-4636-bc4d-99fffe84479e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.787033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e2391-fe54-4636-bc4d-99fffe84479e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.787570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e2391-fe54-4636-bc4d-99fffe84479e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.885770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5029d4-a5f6-49ed-98ee-9e8f316efc85 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:12.888847 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e2391-fe54-4636-bc4d-99fffe84479e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:12.891610 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e2391-fe54-4636-bc4d-99fffe84479e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:12.939875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:12.940570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:12.940846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:12.941115 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:12.942635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:12.942934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:12.944345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:12.956079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 222/431] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:21:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:12.962961 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:12.966703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.043576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.045460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 210/432] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:13.047044 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e76b3-865c-45cb-ac6f-923360facedb None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.047330 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e76b3-865c-45cb-ac6f-923360facedb None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.047630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e76b3-865c-45cb-ac6f-923360facedb None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.048925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 211/433] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:13.049764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e76b3-865c-45cb-ac6f-923360facedb None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.094257 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f5f1c-db5b-49ca-be38-78131c414f08 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.130369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.131025 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.131305 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.131576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.163206 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e76b3-865c-45cb-ac6f-923360facedb None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.199850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d83c8-c23a-4bc3-80ac-8e035cb144a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.200725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d83c8-c23a-4bc3-80ac-8e035cb144a4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.201000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d83c8-c23a-4bc3-80ac-8e035cb144a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.201296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d83c8-c23a-4bc3-80ac-8e035cb144a4 None admin] PATH_INFO: `/v3/projects/730c37dd34844384872ab44ea74c7db5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.202220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.202669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d83c8-c23a-4bc3-80ac-8e035cb144a4 None admin] RBAC: Authorizing `identity:get_project(project_id=730c37dd34844384872ab44ea74c7db5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.204005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d83c8-c23a-4bc3-80ac-8e035cb144a4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.205073 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.205742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 212/434] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/projects/730c37dd34844384872ab44ea74c7db5 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:13.206958 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.207238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.207509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.224666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:13.225902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:13.237481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.297795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 213/435] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:13.298868 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.331902 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 223/436] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:21:13.333954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.403936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c18cf-20fe-4ec9-8d3e-9303d181e74b None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.441449 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.441700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab579fe0-5e64-494c-aea7-33e3042195c3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.442230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.442506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.442775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.444633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.445435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.452245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 214/437] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:13.452245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.467938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.483032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.484152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.484705 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.485042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.496883 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:13.505065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:13.506035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:13.506877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 224/438] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:13.507597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.534889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.564933 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05937d7-100c-493a-bddc-9df58cf54a68 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.602979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.603847 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.604128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.604399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.605626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.607093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.613451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 215/439] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:13.613451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.628571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e231f0d-d405-4554-a167-0911a9e5fe80 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.669591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.670255 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.670533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.670805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.671754 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=730c37dd34844384872ab44ea74c7db5) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:13.672214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.673684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.689435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 225/440] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=730c37dd34844384872ab44ea74c7db5&role.id=afd7d26260694a27a2da7aca75966a85 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:13.689435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.709681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.731681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d149b6a-c14a-4b96-9c48-e6199570eb5d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.770691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.771374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.771651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.771915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.772667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:13.772965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.774425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.785165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 216/441] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:13.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.800381 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:13.821401 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3397-60d7-4916-a2d6-8ebf10bd7a88 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.858435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.859321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.859596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.859903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.861133 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:13.862676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:13.869303 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 226/442] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:13.871037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:13.918990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dc419-ddbd-4910-827f-b9b61fd855f6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:13.971677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:13.972258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:13.972541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:13.972870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:13.988477 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.010712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb2a9-c67a-461b-8f5a-d4b7779514fe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.056518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.057182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.057534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.057807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.058645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=9f78fe1c53c24d8ca59e298f2c585211, scope.project.id=29da51f8b3a24b1485f018c1593a7938) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:14.058955 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.060393 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9ff5-fee7-4f53-b1f5-56d2f61afefa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.067870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 227/443] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=9f78fe1c53c24d8ca59e298f2c585211 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:14.074350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.075880 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.114703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8791b-5335-450a-861b-42d106ef6fb5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.115042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8791b-5335-450a-861b-42d106ef6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.115256 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8791b-5335-450a-861b-42d106ef6fb5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.116675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 228/444] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:21:14.122620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.122976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.123322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.141938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:14.143293 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:14.154517 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.211246 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 217/445] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:14.217798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.237873 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 229/446] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:14.261777 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.280009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.345144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53a108-09f1-47da-a538-316c2efa4007 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.381231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee18f8-5926-4627-a4a6-9a55e97b81f2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.392851 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.393775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.394337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.394617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] PATH_INFO: `/v3/projects/29da51f8b3a24b1485f018c1593a7938` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.396004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] RBAC: Authorizing `identity:get_project(project_id=29da51f8b3a24b1485f018c1593a7938)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.397914 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.399679 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 218/447] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/projects/29da51f8b3a24b1485f018c1593a7938 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:14.405981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.419340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.420037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.420314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.420581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.431065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:14.439372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:14.440299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:14.441132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 230/448] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:14.447366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.452353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.512773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57faf4-b24e-49f8-a892-ba65827d8257 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.525912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.547457 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.548313 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.548587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.548853 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] PATH_INFO: `/v3/groups/fdff84090dc546c4919120f59db76122` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.550036 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] RBAC: Authorizing `identity:get_group(group_id=fdff84090dc546c4919120f59db76122)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.551312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0859-3dba-41c9-ab90-8226d5f483f9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.551475 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.553090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 219/449] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/groups/fdff84090dc546c4919120f59db76122 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:14.558874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.584237 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.584905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.585155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.585464 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.586224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:14.586547 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.587901 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.595822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 231/450] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:14.601344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.666979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60419d2-06a8-4d13-922c-5e3a1a02025c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.669774 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.702678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.703339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.703676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.703951 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.704788 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] RBAC: Adding query filter params (group.id=fdff84090dc546c4919120f59db76122, role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=29da51f8b3a24b1485f018c1593a7938) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:14.705099 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.706578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.708511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629f18d-0ec9-4343-adcc-7669f706bff0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:14.712384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 220/451] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/role_assignments?group.id=fdff84090dc546c4919120f59db76122&scope.project.id=29da51f8b3a24b1485f018c1593a7938&role.id=afd7d26260694a27a2da7aca75966a85 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:14.743760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:14.744440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:14.744716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:14.744986 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:14.816239 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:14.817650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:14.885480 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:14.926899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 232/452] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:21:14.932784 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:14.982812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:15.041878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4743a5-a147-4267-9520-f899097629d0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:15.080357 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:15.081786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:15.081786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:15.081786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] PATH_INFO: `/v3/projects/b58c8de9649042479ca5651859a02fd6` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:15.082925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] RBAC: Authorizing `identity:get_project(project_id=b58c8de9649042479ca5651859a02fd6)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:15.084388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:15.086294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 221/453] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:21:14 2022] GET /identity/v3/projects/b58c8de9649042479ca5651859a02fd6 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:15.092267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:15.195862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007a595-42d5-4a11-89fc-dd1249f7393b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:15.230351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31387a-63ba-4d87-807c-0d744fbf399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:15.231185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31387a-63ba-4d87-807c-0d744fbf399d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:15.231458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31387a-63ba-4d87-807c-0d744fbf399d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:15.231721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31387a-63ba-4d87-807c-0d744fbf399d None admin] PATH_INFO: `/v3/groups/b635485920c1435e8f03f97b11ca442d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:15.232918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31387a-63ba-4d87-807c-0d744fbf399d None admin] RBAC: Authorizing `identity:get_group(group_id=b635485920c1435e8f03f97b11ca442d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:15.234379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31387a-63ba-4d87-807c-0d744fbf399d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:15.235984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 233/454] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:21:15 2022] GET /identity/v3/groups/b635485920c1435e8f03f97b11ca442d => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:15.242572 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:15.345090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd2e2f-0a81-4c37-88c4-74a2652cc618 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:15.358004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:15.380742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:15.381429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:15.381707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:15.381983 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:15.382849 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] RBAC: Adding query filter params (group.id=b635485920c1435e8f03f97b11ca442d, role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=b58c8de9649042479ca5651859a02fd6) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:15.383156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:15.384627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14470759-b5f8-4088-aa7b-9ae2f6ba1058 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:15.391777 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 222/455] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:21:15 2022] GET /identity/v3/role_assignments?group.id=b635485920c1435e8f03f97b11ca442d&scope.project.id=b58c8de9649042479ca5651859a02fd6&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:18.029832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93c955-0e83-4795-948d-1603547478a9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.030419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93c955-0e83-4795-948d-1603547478a9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.030419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93c955-0e83-4795-948d-1603547478a9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.032043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 234/456] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:18.037182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629ed48-5bf8-49fd-824f-c51536cef2c4 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.037639 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629ed48-5bf8-49fd-824f-c51536cef2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.037842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629ed48-5bf8-49fd-824f-c51536cef2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.057284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-9629ed48-5bf8-49fd-824f-c51536cef2c4 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:18.058481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-9629ed48-5bf8-49fd-824f-c51536cef2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:18.152456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 223/457] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:18.162674 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283894da-897f-4918-87bd-c3a3dad82d67 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.162948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283894da-897f-4918-87bd-c3a3dad82d67 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.163228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283894da-897f-4918-87bd-c3a3dad82d67 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.164851 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 235/458] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:18.170132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.170425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.170707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.180023 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283894da-897f-4918-87bd-c3a3dad82d67 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.198952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:18.200053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:18.261639 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:18.298657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 224/459] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:18.327826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.363292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283894da-897f-4918-87bd-c3a3dad82d67 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:18.376909 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:18.438873 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:18.440085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.440662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.441128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.447039 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a1820-0a6a-4266-a632-acb5b96214c6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:18.470629 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:18.472444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:18.484440 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:18.485356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.485636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.485904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.497029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:18.498536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:18.511277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:18.512263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:18.513844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:18.513844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:18.513844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:18.513844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:18.513844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 236/460] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:18.517746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:18.518569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:18.520221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:18.520221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:18.520221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:18.520221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:18.520692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.520836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 225/461] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:18.524472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.678000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:18.683391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:18.698769 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7583125-2c5a-442d-9399-362597393b20 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:18.699862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f761fb9-e4fe-4a02-9b52-b3514c4194fd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:18.741750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:18.742530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.742869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.743207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.744029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:18.744466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:18.745037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:18.745954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.746077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dc6b6-02e9-449b-9360-dd92912dc008 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:18.746520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.746969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.747886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:18.748355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:18.749957 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:18.761858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 226/462] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:18.763272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 237/463] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:18.763403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844111a-831a-48cf-975c-0cf330fc6c04 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.763656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844111a-831a-48cf-975c-0cf330fc6c04 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.763990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844111a-831a-48cf-975c-0cf330fc6c04 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.769375 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 227/464] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:18.769498 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.771447 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.771728 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.771999 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.794917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:18.796462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:18.877341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 228/465] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:18.878600 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.894515 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec0f91-57cf-48d4-b274-affbe1edd68b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:18.931749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:18.932779 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:18.933309 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:18.933771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:18.934689 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:18.936373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:18.960226 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 238/466] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:21:18 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:21:18.961392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:18.964305 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:18.984433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:19.013324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e1d28-c07a-47ee-a379-34f971e22e23 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.059510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3dcab0-f59d-4f99-83ae-f45af29723b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.062364 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3dcab0-f59d-4f99-83ae-f45af29723b4 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.062827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3dcab0-f59d-4f99-83ae-f45af29723b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.063324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3dcab0-f59d-4f99-83ae-f45af29723b4 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.064254 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3dcab0-f59d-4f99-83ae-f45af29723b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.065953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3dcab0-f59d-4f99-83ae-f45af29723b4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.093682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 229/467] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:21:18 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:19.095342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56be8f-d4b9-436f-b820-610c50fa5f36 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.095636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56be8f-d4b9-436f-b820-610c50fa5f36 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.095916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56be8f-d4b9-436f-b820-610c50fa5f36 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.096987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbc549-119e-493c-88f3-55b050f08007 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.101439 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 230/468] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:19.104214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.104500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.104782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.126319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:19.127899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:19.140176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.141381 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.141818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.142243 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.155009 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.156886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:19.176521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:19.177601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:19.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:19.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:19.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:19.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:19.180685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 239/469] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:19.186961 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:19.214592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 231/470] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:19.240679 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:19.279975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:19.320121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:19.334804 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2023a5-8070-4bec-86ec-ed1ad0806621 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.361506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835af7c5-52ed-411a-815a-41c61f308cb1 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.371258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.372067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.372347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.372613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.373371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:19.373671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.375037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.393342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 240/471] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:19.394932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:19.402946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.403917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.404631 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.404964 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.417282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.419419 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:19.442883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:19.443910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:19.444783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:19.444783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:19.444783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:19.444783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:19.445012 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 232/472] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:19.450723 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:19.511422 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9cdca-0dba-444a-9ba0-ffa95e7c52ef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.526230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:19.549748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b0518-2766-418e-8684-7f7082305a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.550340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b0518-2766-418e-8684-7f7082305a11 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.550616 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b0518-2766-418e-8684-7f7082305a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.550886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b0518-2766-418e-8684-7f7082305a11 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.551591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b0518-2766-418e-8684-7f7082305a11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.553024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b0518-2766-418e-8684-7f7082305a11 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.573009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ec14f-124c-485a-a45c-55e8830d177a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.584832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 241/473] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:21:19 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:19.589988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f648c-ed12-444f-bedd-43cc0dd23d4a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.591152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f648c-ed12-444f-bedd-43cc0dd23d4a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.592206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f648c-ed12-444f-bedd-43cc0dd23d4a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.594294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 242/474] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:19.601334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.601334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.601334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.616615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:19.636110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:19.638949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:19.656505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.658390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.658655 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.658937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.659745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:19.660057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.661932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.689134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 233/475] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:19.710448 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:19.733227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 243/476] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:19.755692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:19.879020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:19.884120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e61dc-929d-4bb6-a96e-029fdabbaed4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.909128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8995d-7e07-446e-b482-5071e5b24e26 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:19.943884 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.944579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.944855 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.945124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.945951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.947351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.954606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.sql.core [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:21:19.954606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:21:19.954606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [parameters: {'id': '6e55d132e8554b728e675194b2ca645d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:21:19.954606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80298) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:21:19.963856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:19.964947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:19.965459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:19.965950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:19.966893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:19.968599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() May 19 14:21:19.972968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:21:19.973898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application yield resource May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise self.value May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application session.commit() May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:21:19.974890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.flush() May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application compat.raise_( May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:21:19.975791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:21:19.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:21:19.979061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [parameters: {'id': '6e55d132e8554b728e675194b2ca645d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:19.981198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:21:19.983176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:19.985275 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 244/477] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:21:19 2022] POST /identity/v3/roles => generated 144 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:19.985826 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:20.006022 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 234/478] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:21:19 2022] POST /identity/v3/users => generated 249 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:20.105199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84e18-3193-4910-88a0-0aa3236b0b0c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:20.142263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa469-2a18-417e-b6f9-c88b84a11a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:20.143007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa469-2a18-417e-b6f9-c88b84a11a88 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:20.143283 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa469-2a18-417e-b6f9-c88b84a11a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:20.143556 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa469-2a18-417e-b6f9-c88b84a11a88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:20.155183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2aa469-2a18-417e-b6f9-c88b84a11a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5a2aa469-2a18-417e-b6f9-c88b84a11a88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:20.165594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:20.166581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:21:20.167440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:20.168083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 245/479] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:20.174350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:20.226612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:20.304992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3f779-4f17-4ef5-a654-6b0b08aeaf43 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:20.340318 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:20.341001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:20.341304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:20.341581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:20.342337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:20.342705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:20.344203 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f097-f356-4283-a05a-6be179be77d0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:20.352729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 235/480] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:21.753375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828508-0460-41e5-b42f-6a0c2a2103c5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:21.753916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828508-0460-41e5-b42f-6a0c2a2103c5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:21.753916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828508-0460-41e5-b42f-6a0c2a2103c5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:21.755685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 246/481] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:21.761826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0c613-9e02-41f2-82e7-1e132da03599 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:21.762347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0c613-9e02-41f2-82e7-1e132da03599 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:21.762347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0c613-9e02-41f2-82e7-1e132da03599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:21.790324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-ecf0c613-9e02-41f2-82e7-1e132da03599 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:21.791907 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ecf0c613-9e02-41f2-82e7-1e132da03599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:21.867758 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 236/482] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:21.891082 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828508-0460-41e5-b42f-6a0c2a2103c5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:21.938171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.002724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf61db-83cb-45a3-81be-69d13dcb5b4b None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.003362 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf61db-83cb-45a3-81be-69d13dcb5b4b None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.003887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf61db-83cb-45a3-81be-69d13dcb5b4b None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.005512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 237/483] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:21:22.010676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.010958 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.011232 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.039228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:22.041399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:22.049191 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828508-0460-41e5-b42f-6a0c2a2103c5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.068353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.120303 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 238/484] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:22.135701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.136855 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.137394 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.137857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.142814 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.166224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:22.183641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:22.184632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:22.185486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 247/485] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:22.193518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.325511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9f0c1-1806-46e2-9ce7-37d812a77d90 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.355693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.377557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc000a8-2552-4eb4-b5f5-ef8cce56a512 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.391300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed13a00-2ee7-4420-8e9b-bd201087898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.391950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed13a00-2ee7-4420-8e9b-bd201087898c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.392228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed13a00-2ee7-4420-8e9b-bd201087898c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.392496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed13a00-2ee7-4420-8e9b-bd201087898c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.412330 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed13a00-2ee7-4420-8e9b-bd201087898c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-2ed13a00-2ee7-4420-8e9b-bd201087898c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:22.420117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:22.421169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:22.422057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:22.424769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 239/486] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:22.426216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783bf70-f8b5-4ecc-96fe-5237efc66c58 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.426498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783bf70-f8b5-4ecc-96fe-5237efc66c58 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.426772 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783bf70-f8b5-4ecc-96fe-5237efc66c58 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.428287 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 240/487] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:22.432606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783bf70-f8b5-4ecc-96fe-5237efc66c58 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.440494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.441522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.442475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.442944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.443882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:22.444714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:22.446916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6ec14-0cef-431a-ae9e-0c8e2c9e6d1a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:22.458069 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 248/488] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:22.460569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.461406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.466332 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.480795 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:22.482094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:22.545120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783bf70-f8b5-4ecc-96fe-5237efc66c58 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.555116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 249/489] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:22.556007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.590653 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.591815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.592232 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.593405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.594327 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:22.594641 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:22.598346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:22.614585 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 241/490] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:22.614585 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.644024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.674303 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891ad64-abf8-492e-868f-887d17e515ea None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.712583 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.713570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.713958 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.714300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.721657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.729779 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:22.731815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:22.749581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1c2eb-9ca7-45f6-9704-5aca2df0a960 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:22.758867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:22.760147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:22.760987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:22.760987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:22.760987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:22.760987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:22.763506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 250/491] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:22.764985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.792566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.793489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.793823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.794095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.804775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:22.812719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:22.813860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:22.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 242/492] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:22.816410 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.901542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b3f0-9f9d-47d7-96ad-cd6e77a31fe7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.932436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fd4b3-1a78-41f0-9cca-816ce177d5f7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:22.934437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.942903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.943712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.943991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.944260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.956665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:22.958205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:22.971015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:22.971704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.971985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.972257 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.972993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:22.973343 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:22.974804 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8fbf7-bed2-4eb6-8a71-2e5fb4441303 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:22.978245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:22.984365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:22.985671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:22.986543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:22.986543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:22.986543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:22.986543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:22.990725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 251/493] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:22.993300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 243/494] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:22.993445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:22.993544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa48e5-38a7-4026-9d6b-9f23b6a0c090 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:22.993544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa48e5-38a7-4026-9d6b-9f23b6a0c090 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:22.993544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa48e5-38a7-4026-9d6b-9f23b6a0c090 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:22.994579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 244/495] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:22.995270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa48e5-38a7-4026-9d6b-9f23b6a0c090 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.100187 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea624-6f49-4320-a942-c450b710ec4d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.111318 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa48e5-38a7-4026-9d6b-9f23b6a0c090 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.136668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.137367 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.137665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.137923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.138672 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:23.138961 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.140323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.146646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.147317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.147598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.147869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.148590 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:23.148899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.149075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 252/496] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:23.149731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.150300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd47a8-7c01-4154-a8a7-5e517a30ea4e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.164083 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 245/497] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:23.165487 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004297b9-6875-4a38-a291-76ea24e47e25 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.165680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004297b9-6875-4a38-a291-76ea24e47e25 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.165884 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004297b9-6875-4a38-a291-76ea24e47e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.181674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-004297b9-6875-4a38-a291-76ea24e47e25 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:23.182868 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-004297b9-6875-4a38-a291-76ea24e47e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:23.192282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.231299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.247614 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 246/498] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:23.247924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc22561-f184-449f-a886-d14597991bd1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.248961 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084c40a-239a-4081-bfb1-3ad8509f025e None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.249267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084c40a-239a-4081-bfb1-3ad8509f025e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.249540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084c40a-239a-4081-bfb1-3ad8509f025e None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.250805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 247/499] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:23.251478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084c40a-239a-4081-bfb1-3ad8509f025e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.292862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.293830 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.294106 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.294374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.301843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.303309 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:23.318403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:23.319359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 253/500] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:23.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.357067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084c40a-239a-4081-bfb1-3ad8509f025e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.389689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.390496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.391076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.391076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.401260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.402724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.416723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275519c4-5c2d-4a63-bcae-0be989db840b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.418202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-c7e9d2d0-3a4e-4bc4-86a0-9a58d5879e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:23.420511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:23.421505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:23.422482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:23.422482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:23.422482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:23.422482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:23.422482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 248/501] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:23.423282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.423561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.423834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.441676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:23.442851 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:23.451042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.451847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.452120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.452384 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.460204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.461699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.471081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:23.475846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:23.476969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:23.477904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 254/502] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:23.478470 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.516865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 249/503] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:23.517820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.593867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed3ce3-ee13-4ed8-8d40-1aa74d814656 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.619378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4cf8f-b0e1-4224-8e54-f940f640c665 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.628961 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.629798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.630075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.630340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.640403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:23.648321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:23.649296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:23.650159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:23.654735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 255/504] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:23.655540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.657438 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.658116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.658391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.658657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.659488 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:23.659792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.661223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.679866 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.701866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 250/505] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:23.702734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.731463 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.763596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0164d4e-a03e-4bd3-981a-e1c3cae9a967 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.798361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.799529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.799809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.800080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.800796 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:23.801097 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.802594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.804828 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95667163-0d72-4423-b445-df1aa719dbac None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.816650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 256/506] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:23.818117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.839467 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.840266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.840545 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.840820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.841743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:23.842079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.843526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.861530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 251/507] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:23.862456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:23.902872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.928015 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42369c6f-9cd0-4f6b-921c-584c2de7de14 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.959858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:23.964554 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:23.965393 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:23.965674 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:23.965944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:23.975991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:23.976554 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50c8b2-c288-4fcc-ba98-9ac287387507 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:23.977479 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:23.995246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:23.996296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:23.997174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:23.997174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:23.997174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:23.997174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:23.997174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 257/508] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:23.997878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.016855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.017670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.017950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.018221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.018971 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:24.019275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.020678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.029630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 252/509] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:24.030398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.104700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff15abf-7a2a-40db-b9bb-1972476d9683 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.130622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553495-f0ee-4a21-83c0-88a0945cc993 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.138974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.139619 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.139891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.140157 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.142822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.164008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.164812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.165110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.165413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.173148 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.174661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:24.190243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:24.191151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:24.192094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 253/510] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:24.192660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.201729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.220698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.222215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.298755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f495c-189f-476b-9c60-43dc53c87c15 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.326809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 258/511] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:24.326809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.339053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.339715 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.339991 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.340260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.360497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.408080 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.409612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.417428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.432336 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cb5d9-92a7-477c-8ddb-472ef18eb628 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.466153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.466717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.466996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.467265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.467992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:24.468284 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.469755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.481062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 259/512] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:24.481905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.514076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 254/513] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:24.515014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.574742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.582095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ff4af-b3b6-4937-9102-192868ab750c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.615481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.616424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.616767 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.617036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.619476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d3f5-d9eb-4c3c-b997-8c4746e9d76a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.626924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.627669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.628978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:24.646214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:24.647265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:24.653034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:24.653034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:24.653034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:24.653034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:24.653034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 260/514] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:24.653034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.656884 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.657597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.657877 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.658148 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.658993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:24.659306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.660774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.677730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 255/515] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:24.678491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.750494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc81eb-5d9c-4b1b-9255-405a349a7b15 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.773631 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa7859-dfad-4d05-9370-3aad651efd1d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.784276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.785079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.785396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.785669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.796137 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.797716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.804054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.807511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.808135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.808407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.808672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.809401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.810827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:24.815963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:24.816800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:24.817696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:24.817696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:24.817696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:24.817696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:24.817696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 261/516] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:24.817981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.832316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 256/517] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:21:24 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:24.833127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:24.859360 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:24.952352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990ffe5-447b-4150-b3e5-a5ee2af40ae4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.952563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e5952-42da-46d8-bf10-a682009dfe0c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:24.988394 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.989355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.989612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.989916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.991016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:24.991762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:24.992149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:24.992475 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:24.993338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:24.993665 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:24.995116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.002723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.004229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.013426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 257/518] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:25.013426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:25.025419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:25.026315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:25.027576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:25.027576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:25.027576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:25.027576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:25.027576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 262/519] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:25.027576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.092974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.103939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.123878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e032-af64-41a3-b620-66ecefda3242 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.153935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d21eee-21f5-4ec4-b329-417875717534 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.159898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.160569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.160845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.161122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.188729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.189436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.189713 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.190023 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.190738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:25.191041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.192492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.204259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 263/520] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:25.204943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.240764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.242658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.311462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fcd65-1e26-4656-95e7-c8532ce21230 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.323264 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.327500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.349280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.349930 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 258/521] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:25.350258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.350606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.350952 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.355014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.358865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.360287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:25.375236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:25.376190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:25.377679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 264/522] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:25.379472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.459374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0236-a6d3-4ee3-8ff5-011a902d781a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.494963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.495633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.495764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec640b-1b66-4d46-a851-87ea69cdfdfb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.495949 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.496189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.496906 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:25.497217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.498642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.510322 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 259/523] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:25.515306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.531620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.532466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.532752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.533032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.540138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.541633 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:25.555959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:25.556829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 265/524] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:25.558499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.564256 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.591441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.626130 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edecad3-b9ca-4700-83bf-38bd730865c5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.661042 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.661980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.662259 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.662525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.667495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266467b-6d0a-4baf-8c84-533cae58cb11 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.674253 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.675733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:25.692670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:25.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:25.694466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:25.694466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:25.694466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:25.694466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:25.694466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 260/525] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:25.694764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.709792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.710477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.710761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.711024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.711875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:25.712181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.713667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.757397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 266/526] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:25.757397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.794068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.804112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5b48b-afcc-49d5-a6ec-f72289db1c66 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.837006 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.837762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.838040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.838308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.845053 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:25.845527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.846759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.859991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97781c44-6cdf-490f-ad46-b842c78f413b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:25.861455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:25.862497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:25.863416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 261/527] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:25.863892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.900395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:25.901156 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:25.901470 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:25.901747 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:25.902538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:25.902851 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:25.904664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:25.925877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 267/528] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:25.930140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:25.973275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678db08b-ba3a-494d-b734-9be0095629d2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.008945 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.009645 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.009924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.010204 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.010925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.011231 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.012627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.028123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 262/529] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:26.028123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.038906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c2628-5ce9-47f7-85a0-f8a1f04c5aee None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.044584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.076399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.077330 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.077613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.077879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.096110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.132982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f42f-9cf2-4bc1-858a-73e6d9bd840d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.145905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.147366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.168791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.169654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.169937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.170212 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.171004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.171315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.172842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.190571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 263/530] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:26.191400 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.269142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 268/531] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:26.269142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.272900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.302902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008868f-068b-444c-ba1b-23dc602e8720 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.316928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.339990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.340803 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.341842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.342124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.342957 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=4528a20cb3b24aaab0ff2b52b56da1c6) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.343266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.344772 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.361284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 264/532] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/role_assignments?user.id=4528a20cb3b24aaab0ff2b52b56da1c6&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:26.361284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.373371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086856d0-cabf-4d10-bee7-5a2514216e82 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.412350 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.413210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.413511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.413781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.424166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.425942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:26.445530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:26.446514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:26.447395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:26.447818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 269/533] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:26.448330 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076c654-8824-4aa6-a84a-c51cf9e3bd6d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.550802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.550802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.550802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.551351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.552144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=b4d24bc13b5c469cade02763acf072bd) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.552522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.554682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.559141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.563904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 265/534] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/role_assignments?user.id=b4d24bc13b5c469cade02763acf072bd&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:26.565445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.591204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc667321-a11f-4341-8736-41cbefa5e7bc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.603516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.654497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.655263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.655522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.655788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.682561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.684072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.707458 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ea51-a67f-412a-986a-3e4363217809 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:26.722127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:26.723262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:26.724220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:26.724220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:26.724220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:26.724220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:26.724220 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 270/535] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:26.739901 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.756013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.757057 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.757526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.757937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.759655 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.760159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.761845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.778009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 266/536] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:26.784454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.849221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.863128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bae3e1-447c-459a-8060-9cbb2c0b5403 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.894661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557d18-d676-4a89-8d03-c856938301cc None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:26.898632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.899415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.899793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.900162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.900998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.901424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.902912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.918351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 271/537] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:26.929265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:26.944352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:26.971927 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:26.973002 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:26.973524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:26.974037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:26.975288 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=de43fcf2258347a797867a32d8a7b1a3) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:26.975829 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:26.977549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:26.990632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 267/538] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/role_assignments?user.id=de43fcf2258347a797867a32d8a7b1a3&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:27.032788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb3ede-ea4d-4d8f-bd85-7a72a6aeeff9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:27.069897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:27.070729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.071004 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.071277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.078578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:27.080081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a399820d-4672-4847-b06c-324c5afe2658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:27.095130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:27.096061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:27.096912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 272/539] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:27.102411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:27.178230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f870e4-57aa-4789-a7a9-2b49c4463576 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.178427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f870e4-57aa-4789-a7a9-2b49c4463576 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.178682 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f870e4-57aa-4789-a7a9-2b49c4463576 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.180399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 273/540] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:27.186453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.186755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.187016 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.212727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:27.214019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:27.262147 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3438d7-c811-48d0-92c8-5b0e7254d52c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:27.291429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:27.304415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:27.305091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.305407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.305689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.306486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:27.306797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:27.307907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 274/541] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:27.308249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:27.321353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 268/542] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:27.326935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:27.331355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:27.339895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:27.453105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e45ed3-2465-4554-8132-70042e38c006 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:27.489350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:27.490023 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.490311 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.490592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.491588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=d8507240870340478b50c27ae54381ef) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:27.492029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:27.493688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:27.500724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 269/543] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:27 2022] GET /identity/v3/role_assignments?user.id=d8507240870340478b50c27ae54381ef&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:27.548358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6276ff5-4161-41e3-9df6-c0070cc90e66 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:27.619472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:27.620306 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.620587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.620858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.642197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:27.650694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:27.651676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:27.652523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 275/544] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:27.658657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:27.748000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:27.819257 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1d245-d81d-49a4-88de-fd44a0e4497f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:27.863087 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:27.863878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:27.864036 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:27.864290 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:27.865044 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:27.865386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:27.866781 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:27.880867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 270/545] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:27.886311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:28.039616 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:28.087891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43b67-0a3b-498a-b855-49a26e972637 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:28.136693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:28.137540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:28.137821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:28.138195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:28.150444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:28.151929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:28.170034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:28.170973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:28.171837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:28.171837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:28.171837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:28.171837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:28.171837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 276/546] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:28.177077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:28.310045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce92039-8979-4550-ba3d-350a0060dd20 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:28.333649 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:28.352718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:28.353433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:28.353711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:28.353983 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:28.354705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:28.355009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:28.356396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:28.373219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 271/547] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:28.374622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:28.494853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435d4e5-535d-44f6-9fb3-236286605328 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:28.522049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:28.531226 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:28.532844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:28.533407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:28.534075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:28.541914 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:28.543369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:28.557604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:28.559010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:28.560152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:28.563488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 277/548] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:28.564938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:28.667634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83c059-85d1-4564-9ad2-11bc5bb28718 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:28.706630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:28.707328 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:28.707615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:28.707889 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:28.708731 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:28.709037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:28.710587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:28.733353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 272/549] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:21:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:28.735006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:28.835635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564658-68b6-4d6d-b0a5-4c72a684b00e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:28.869613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:28.870273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:28.870550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:28.870819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:28.949799 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:28.976675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:28.978166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:29.110577 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 278/550] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:29.116366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.146166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:29.218705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059abe13-ccfd-427b-aadf-f676f27f9699 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:29.252663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:29.253496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.253776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.254046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.264471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:29.266038 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-feb06746-1273-4873-b5b1-ce4735b94aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:29.284893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:29.285953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:29.286931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:29.286931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:29.286931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:29.286931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:29.290936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 273/551] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:29.292416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.306854 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70a5d-fb28-4bd8-9828-0ccb5097fbd3 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.307658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70a5d-fb28-4bd8-9828-0ccb5097fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.308133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70a5d-fb28-4bd8-9828-0ccb5097fbd3 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.310176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 274/552] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:21:29.318152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.318857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.319332 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.338228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:29.339845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:29.398081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:29.433348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c055326-e66c-466b-806a-54e97a0832f3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:29.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 275/553] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:29.472062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.481340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:29.484059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.485696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.486264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:29.487467 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.489538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:29.491551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:29.494128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:29.516595 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 279/554] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:29.521797 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.624382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c666cc3-69a2-4744-a704-3192a57c4dca None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:29.647539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c56b3-1ad2-4663-866d-9692dbabfc86 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:29.666765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65a70b-3bc3-4957-9dca-b1b47dab0fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:29.667935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65a70b-3bc3-4957-9dca-b1b47dab0fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.668438 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65a70b-3bc3-4957-9dca-b1b47dab0fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.668962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65a70b-3bc3-4957-9dca-b1b47dab0fb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.680997 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65a70b-3bc3-4957-9dca-b1b47dab0fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:29.683068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:29.683880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.684159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.684433 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-5a65a70b-3bc3-4957-9dca-b1b47dab0fb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:29.690398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:29.691372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:29.692250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:29.692944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:29.693521 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:29.695395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 276/555] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:29.697075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867a3c1-b13b-4b2c-afc8-f602f8a8d0e1 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.697595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867a3c1-b13b-4b2c-afc8-f602f8a8d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.698060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867a3c1-b13b-4b2c-afc8-f602f8a8d0e1 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.699757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 277/556] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:29.700643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867a3c1-b13b-4b2c-afc8-f602f8a8d0e1 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-29375181-9cd4-41a3-8459-ebab612f3f67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:29.712852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:29.714122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:29.715468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 280/557] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:29.716001 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.716001 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.716560 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.721726 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:29.733615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:29.734923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:29.795015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:29.817555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 281/558] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:29.818478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.841645 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867a3c1-b13b-4b2c-afc8-f602f8a8d0e1 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:29.882564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:29.883735 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.884223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.884725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.885794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:29.886338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:29.887993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:29.897316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 278/559] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:29.898396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:29.927529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab2a0f-0613-4f9c-afaf-2eb2a771c549 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:29.958865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:29.964833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:29.965562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:29.965846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:29.966118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:29.966932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:29.967246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:29.968732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:29.982125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 282/560] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:29.983018 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.021435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6b844-b6ca-42da-86a0-825fb264bce7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.053091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.060300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699d99e-c0be-4853-91b0-8d2dd33acfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.061451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699d99e-c0be-4853-91b0-8d2dd33acfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.061954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699d99e-c0be-4853-91b0-8d2dd33acfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.062433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699d99e-c0be-4853-91b0-8d2dd33acfa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.074359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699d99e-c0be-4853-91b0-8d2dd33acfa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a699d99e-c0be-4853-91b0-8d2dd33acfa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:30.083081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:30.084043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:30.084950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:30.086916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 279/561] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:30.087708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07b95a-d3cb-42c5-977f-04a611961fe8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.088439 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037af6e-7d69-4034-9a61-031b7b76c695 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.088859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037af6e-7d69-4034-9a61-031b7b76c695 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.089309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037af6e-7d69-4034-9a61-031b7b76c695 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.090724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 280/562] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:30.095333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037af6e-7d69-4034-9a61-031b7b76c695 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.126544 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.127348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.127627 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.127900 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.138946 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.140530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:30.159523 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:30.160500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:30.161819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:30.161819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:30.161819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:30.161819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:30.161819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 283/563] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:30.161819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.205307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037af6e-7d69-4034-9a61-031b7b76c695 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.205831 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.242541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.243204 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.243484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.243751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.244584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=427c2d8e507749a4b94b7fe4ea37264d) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:30.244901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.246376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5f339-924b-43b6-88d3-2220237a1165 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.252244 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 281/564] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/role_assignments?user.id=427c2d8e507749a4b94b7fe4ea37264d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:30.253540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.253816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.254105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.266848 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d02c-d64b-4a9e-8e35-332ed2b129cb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.282289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:30.284692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:30.305073 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.305824 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.306105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.306377 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.307212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:30.307522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.308965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea8f2-176d-4171-be1c-d9447eaeb0f8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.318645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 284/565] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:30.324774 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12ac9-10e5-4d15-9657-340ffbf98f20 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.325070 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12ac9-10e5-4d15-9657-340ffbf98f20 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.325373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12ac9-10e5-4d15-9657-340ffbf98f20 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.326796 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 285/566] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:30.327544 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd12ac9-10e5-4d15-9657-340ffbf98f20 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.364424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.383853 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 282/567] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:30.385018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.459601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd12ac9-10e5-4d15-9657-340ffbf98f20 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.467835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.495712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.496481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.496827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.497183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.497304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96bfc5-938d-4efa-983c-b1ea6bc48c94 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.498028 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:30.498389 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.500163 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64551e08-e30b-4dda-98e4-b54877558747 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.516999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 286/568] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:30.516999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.516999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.516999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.531868 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:30.533212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:30.537330 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.538474 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.538969 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.539453 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.553898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.556122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:30.575887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:30.576866 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:30.577826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:30.577826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:30.577826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:30.577826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:30.577826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 283/569] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:30.577826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.606451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 287/570] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:30.607565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.633754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.701899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637ae3-0e87-4788-a1af-e8663f615125 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.726148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.734129 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c99e6-78d1-400c-9e38-c76918148870 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.741517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.742483 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.742791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.743131 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.754695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:30.763062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:30.764192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:30.765165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 284/571] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:30.766066 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.775430 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.776351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.776729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.777092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.785148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.786750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:30.801277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:30.802461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:21:30.803664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:30.804375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 288/572] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:30.809359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.870546 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:30.874823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00cf20-622a-413d-9a78-8a5afaf04b83 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.909893 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540924fb-2907-437e-a69e-82abd84f8979 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:30.910513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.911193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.911480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.911755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.912486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:30.912794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:30.914231 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:30.927099 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 285/573] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:30.927935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:30.958419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:30.959366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:30.959928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:30.959928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:30.970966 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:30.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:30.980361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 289/574] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:30.981229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.001385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.039447 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f05dc9-cfa1-470b-99f5-5311b5d8a50c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.077512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.078214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.078614 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.078745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.079501 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:31.080075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.081187 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.089770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 286/575] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:31.094896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.096280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d88e0-88a5-44e1-b249-d47cf8c07544 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.117089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.132927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.133639 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.133920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.134195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.135033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:31.135345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.137196 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.159944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 290/576] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:21:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:31.159944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.201386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9f122-41c8-4455-9e5b-15a366c6cf89 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.237549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.238512 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.238691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.238960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.239238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.249358 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.250849 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.262832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c920baf-9d33-47e0-8068-97f8c78ecc5a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:31.266597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:31.267727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:31.268932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:31.274652 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 287/577] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:31.274652 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.301489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.302248 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.302589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.302933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.303726 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:31.304085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.305552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.317947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 291/578] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:31.317947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.364358 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.384963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5163a7-b337-4abb-bef0-51c583a07a1c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.421212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98570c0a-d2e3-4a4e-beb2-d693f7244eb1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.423846 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.424750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.425123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.425515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.437935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.439701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.457478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.458277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.458608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.458910 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:31.459045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:31.460055 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:31.461175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:31.461175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:31.461175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.461175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:31.462397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 288/579] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:31.463289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.468649 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.479195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.480645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:31.502118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:31.503217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:31.504656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:31.504656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:31.504656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.504656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:31.504656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 292/580] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:31.508345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.573161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d45f0c-726a-44d0-9a1a-e96a21dc018c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.603967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.610374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.611050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.611331 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.611598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.612429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:31.612735 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.614207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.615594 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dcce6-9f9b-44ba-ae23-dce8532a6938 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.631723 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 289/581] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:31.632707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.655754 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.656637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.656916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.657181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.668267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.669750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:31.687120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:31.688135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:31.689058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:31.689058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:31.689058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.689058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:31.689058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 293/582] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:31.690168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.731707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.744850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05008d06-c850-44e3-a3d9-d6ef06ad5fa5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.783135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.783727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.784004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.784274 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.785026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:31.785369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.786754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.802746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 290/583] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:31.802746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.807142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6d28b-58bc-40a2-bab7-a724652bcd15 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.841794 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.842461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.842737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.843003 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.843719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:31.844150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.845555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.852826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.863201 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 294/584] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:31.863201 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:31.915673 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c9dfe-a431-4542-b772-dd55ae0caac3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.957376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:31.958184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:31.958460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:31.958726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:31.969017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:31.970512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:31.973451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:31.977577 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537aa9e-180b-4648-8daa-e9290a8d5ac2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:31.988518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:31.989490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:31.990397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:31.990397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:31.990397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:31.990397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:31.990397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 291/585] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:31.990693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.019695 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.020473 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.020738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.021002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.021780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:32.022078 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.023717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.036462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 295/586] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:32.037280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.096484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee1b8b-96b1-4843-9b56-0507e57976fd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.098104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.134708 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.135639 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.135909 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.136192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.143971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10df43-af9b-4d28-9998-a8036cbb8342 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.144524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.146017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:32.161815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:32.162962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:21:32.164117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:32.167567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 292/587] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:32.168447 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.183819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.184915 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.185216 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.185511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.193419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.194874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:32.210569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:32.211485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:32.212397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 296/588] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:32.212978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.238492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.286952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5aa8d-e65f-4725-aff1-d677cdd8755f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.324210 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.324898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.325182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.325483 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.326211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:32.326518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.327908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.328798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c56c6-d59d-460b-a60f-a84c379e9c31 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.344428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 293/589] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:32.344428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.368809 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.369531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.370345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.370629 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.370918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.380173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.381680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:32.396725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:32.397742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 297/590] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:32.399287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.455917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616e3c-2d93-4fab-aa1a-170fa6b38663 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.479205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.489331 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.490014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.490297 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.490574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.491413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:32.491729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.493184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.497999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9fca-eeb3-48c6-bfe0-dfb465cbe87d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.509844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 294/591] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:32.510646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.532413 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.533080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.533415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.533731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.534533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:32.534843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.536273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.547722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 298/592] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:32.548438 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.588666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.608568 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8ba9c-b764-4c71-bac5-949f84b9f806 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.640635 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.641526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.641808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.642080 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.645557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fcbd4-b6b7-432d-8d33-e300b47ad626 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.649115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.650604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-c5b902e4-9612-4f28-abe4-6d5926cfee8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:32.664661 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:32.665623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:32.666527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 295/593] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:32.667116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa06fe-2abf-4300-90f6-34847963a187 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.667573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa06fe-2abf-4300-90f6-34847963a187 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.667678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa06fe-2abf-4300-90f6-34847963a187 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.669030 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 296/594] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:32.669563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa06fe-2abf-4300-90f6-34847963a187 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.684861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.685601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.685890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.686142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.687001 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:32.687311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.688778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.698235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.706567 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 299/595] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:32.707366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.770390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa06fe-2abf-4300-90f6-34847963a187 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.803662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.804396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.804673 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.804943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.805924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0781-cf20-4260-a024-4c36012ee42d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.813469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.816296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.818524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:32.836010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:32.837069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:32.838196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:32.838196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:32.838196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:32.838196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:32.838196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 297/596] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:32.838196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.842987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf34398-e7c1-4b90-91e2-46e1c53c9371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.844040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf34398-e7c1-4b90-91e2-46e1c53c9371 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.844320 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf34398-e7c1-4b90-91e2-46e1c53c9371 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.844589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf34398-e7c1-4b90-91e2-46e1c53c9371 None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/4528a20cb3b24aaab0ff2b52b56da1c6/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.859108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf34398-e7c1-4b90-91e2-46e1c53c9371 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=4528a20cb3b24aaab0ff2b52b56da1c6, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.860613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf34398-e7c1-4b90-91e2-46e1c53c9371 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:32.874614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 300/597] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:32 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/4528a20cb3b24aaab0ff2b52b56da1c6/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:32.875978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.876260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.876534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.896289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:32.897632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:32.946973 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ee80f-dddf-4a20-aee8-ee2acb45bc11 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:32.957718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:32.984841 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 301/598] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:32.985745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:32.986213 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:32.987257 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:32.987744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:32.988243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:32.989132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:32.989530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:32.991316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.005854 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 298/599] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:33.011801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.076638 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.101982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43013932-a344-4384-80de-13bb7fcf93aa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.124310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0af2b9-7259-46a8-850c-927d51319a69 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.137083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.137918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.138197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.138458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.149054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.150538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.159481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.160422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.160905 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.161380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.162283 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:33.162781 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.164342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:33.168973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:33.169926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:33.170885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:33.170885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:33.170885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:33.170885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:33.170885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 302/600] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:33.171524 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.179720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 299/601] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:33.179720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.207093 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.277271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479720c-87d3-4e72-b979-e4da255955d4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.286270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a407-9908-43f1-a31e-ca8e18695389 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.299911 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.312470 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.313457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.313734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.314029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.324225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:33.324376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.325220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.325524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.325794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/b4d24bc13b5c469cade02763acf072bd/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:33.331979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:33.332920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:33.333834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:33.337798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 303/602] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:33.338992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.341342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=b4d24bc13b5c469cade02763acf072bd, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.342817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.357900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 300/603] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:33 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/b4d24bc13b5c469cade02763acf072bd/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:33.358717 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.483539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.483998 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a897e34-a20a-43f2-8a84-4930471e3ba8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.498763 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7ca5e-4087-49fd-9c8c-afb64f38b5b7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.523165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.524043 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.524329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.524603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.533193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.535111 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.544759 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.545476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.545813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.546086 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.546927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:33.547266 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.548667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:33.555050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:33.556011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:33.556915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:33.559855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 301/604] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:33.561378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.564150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 304/605] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:33.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.591273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.686534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a40f2-f954-4507-9561-5ef1f39ea9a0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.694846 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fafad-33ed-4fcd-ba56-51439056b2e6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.724851 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.725727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.726002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.726274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.733560 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.735209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.737845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.738869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.739141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.739413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.740167 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:33.740470 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.742177 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:33.749782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:33.750751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:33.751661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:33.752121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 302/606] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:33.752121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.755939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 305/607] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:33.757486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.772387 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.856681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:33.866223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719de5-ce0c-42b5-a0ea-61050d68f3f2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.875855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691014b-2873-4f3d-a524-71990b4d7210 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:33.900909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.901749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.902056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.902493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.913836 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.914430 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:33.915289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.915557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:33.916149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:33.916592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:33.917708 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:33.918309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:33.919982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:33.936119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:33.937071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:33.938116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:33.938116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:33.938116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:33.938116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:33.938116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 306/608] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:33.938116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:33.943016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 303/609] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:33.944102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:34.010221 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:34.052302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78dfb8-98af-4a0b-8cad-8a17980e2100 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.060885 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066be1a8-1143-4afb-8bdb-5892aa8d6896 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.086758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.087458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.087741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.088014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.088807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:34.089115 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.090677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.097838 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.098618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.098902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.099173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.099898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:34.100481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.101910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.103882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 307/610] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:34.104709 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:34.119538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 304/611] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:34.120413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:34.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:34.210370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15529beb-f021-4503-a668-100cc6d58dcc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.231124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad79c5-917c-4865-a1ab-2ad614d0ed7e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.234658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:34.246569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.247559 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.247932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.248277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/de43fcf2258347a797867a32d8a7b1a3/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.263563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=de43fcf2258347a797867a32d8a7b1a3, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.265116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.272611 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.273955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.274433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.274982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/d8507240870340478b50c27ae54381ef/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.284670 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 308/612] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:34 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/de43fcf2258347a797867a32d8a7b1a3/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:34.285967 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:34.306863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=d8507240870340478b50c27ae54381ef, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.308444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.337513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 305/613] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:34 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/d8507240870340478b50c27ae54381ef/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:34.403338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c12c-6fe5-4309-9936-fc023a2c202e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.443440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.444585 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.445092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.445558 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.454615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.456494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:34.473601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:34.474644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:34.475609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:34.478319 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 309/614] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:34.485947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:34.581745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:34.652966 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31663e3e-6ace-4115-aa24-c10d807280e2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.699153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.699760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.700037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.700308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.701161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:34.701517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.702982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.725759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 306/615] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:21:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:34.731080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:34.768051 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:34.853089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53f53-5d9f-4f74-bb51-25288c832e7f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:34.888066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:34.888876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:34.889160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:34.889463 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:34.899928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:34.901515 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:34.920020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:34.920963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:34.921972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:34.921972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:34.921972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:34.921972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:34.921972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 310/616] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:34.926993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.064388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249deda-a986-4b48-b7e7-d4fd2e731867 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.102358 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:35.112946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:35.113789 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.114136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.114672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.115415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:35.115720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:35.117114 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b874e-c193-4477-b35c-ab3f355bddab None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:35.133533 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 307/617] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:35.134561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.172228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c634a5-4184-42ae-ac94-2122118c571a None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.172557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c634a5-4184-42ae-ac94-2122118c571a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.172782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c634a5-4184-42ae-ac94-2122118c571a None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.174437 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 308/618] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:35.182459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.182755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.183000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.201629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:35.203580 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:35.281517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 309/619] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:35.302654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.355210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:35.396459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d8a87-64ea-45bc-93ae-626327e8e090 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.412583 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379957c0-08bd-47d7-bcd0-3f09addeeb88 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.421570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:35.470004 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:35.472070 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.472882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:35.473406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.473891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.474892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.476104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.477579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.488466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:35.490858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:35.501706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:35.510004 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:35.510944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:35.511801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:35.521489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:35.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:35.523347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:35.529437 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 310/620] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:35.531184 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 311/621] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:35.537011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.550108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.700329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23ddb4-eb13-4886-a325-90e5f418aa13 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.734490 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f17da-ceca-4339-830f-5f8416497d02 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.769990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:35.770672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.770948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.771223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.772018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:35.772314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:35.773793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:35.803174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 311/622] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:35.810472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:35.811180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.811459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.811730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.812589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:35.813030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:35.814310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.814591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:35.827635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 312/623] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:35.836426 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:35.847954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:35.848535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:35.931981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e97ea-141a-42ed-9755-ba84be3d9f91 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.971701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13045c-ede6-44b6-879a-e1d1202e12df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:35.972363 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce5efe-506c-483c-8771-5fbcb8a8d333 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:35.972571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13045c-ede6-44b6-879a-e1d1202e12df None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:35.972862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13045c-ede6-44b6-879a-e1d1202e12df None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:35.973133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13045c-ede6-44b6-879a-e1d1202e12df None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/427c2d8e507749a4b94b7fe4ea37264d/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:35.991194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13045c-ede6-44b6-879a-e1d1202e12df None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=427c2d8e507749a4b94b7fe4ea37264d, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:35.993278 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13045c-ede6-44b6-879a-e1d1202e12df None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.010167 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 312/624] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:35 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/427c2d8e507749a4b94b7fe4ea37264d/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:36.010344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.011143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.011420 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.011751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.012647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328d88-8250-4602-bc95-c9092088d184 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.013192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328d88-8250-4602-bc95-c9092088d184 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.013699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328d88-8250-4602-bc95-c9092088d184 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.016630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 313/625] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:36.025412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.025412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.025412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.035014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:36.036896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.054828 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:36.056448 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:36.058451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:36.059366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:36.060259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:36.060259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:36.060259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:36.060259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:36.066853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 313/626] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:36.074357 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.138091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 314/627] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:36.176266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.185567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:36.245828 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:36.248704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670298bc-32f7-4318-8b66-752ac2d58495 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.290473 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.291309 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.291683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.292046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.292871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:36.293392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:36.294843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.308228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 314/628] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:36.313707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.322547 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44444d-9b79-4921-9690-7071ccf52ede None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.402110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.402937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.403205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.403472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.414197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d99f-260b-4494-abc9-82eb5b5e50f2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.426179 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:36.443014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:36.444115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:36.444984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:36.445692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:36.447986 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 315/629] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:36.449950 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.450856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.451132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.451449 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.455072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.460135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:36.461637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:36.478165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:36.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:36.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 315/630] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:36.488035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.555653 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:36.604458 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed83db-f8cf-4ec3-95e5-5d95183deb33 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.640230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.641107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.641416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.641695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.642445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:36.642747 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:36.644587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.652894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625f74-72bf-4093-8eb3-55cc9c3b56c6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.654628 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 316/631] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:36.660190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.701991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.702694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.702975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.703324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.704231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:36.704542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:36.706043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.724788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 316/632] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:36.734283 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.772070 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:36.802669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c15e18-77c6-40e6-9480-003752cd09c0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.847886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:36.868199 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.869016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.869311 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.869588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:36.884747 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:36.886636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:36.903513 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da011f72-07a6-4fa9-a5a2-2bd5a782c76b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:36.927159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:36.928149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:36.929073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:36.929073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:36.929073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:36.929073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:36.945264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 317/633] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:36.946840 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:36.979099 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e001-c755-4683-9a5d-2f06f10beaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:36.979756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e001-c755-4683-9a5d-2f06f10beaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:36.980029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e001-c755-4683-9a5d-2f06f10beaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:36.980294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e001-c755-4683-9a5d-2f06f10beaa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.102032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e001-c755-4683-9a5d-2f06f10beaa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.103563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e001-c755-4683-9a5d-2f06f10beaa0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.108885 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df258-9d82-472e-9312-d3dc788f6677 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.140751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.145102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.146094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.146591 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.147063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.148001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:37.148514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.150134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68361821-9203-4700-98f9-ed6c0758aa99 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.163244 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 318/634] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:37.164815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2dd85-a01a-47b8-a104-3c2afcd1fa66 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.165342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2dd85-a01a-47b8-a104-3c2afcd1fa66 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.165843 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2dd85-a01a-47b8-a104-3c2afcd1fa66 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.167524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 319/635] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:37.168687 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2dd85-a01a-47b8-a104-3c2afcd1fa66 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.206769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.217669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 317/636] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:37.220245 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.220702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.221114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.238626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:37.240090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:37.273790 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2dd85-a01a-47b8-a104-3c2afcd1fa66 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.310028 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 318/637] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:37.310560 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.310851 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.311119 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.311397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.312346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.324823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.326821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-730055ba-8784-4220-829d-88024b9392b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:37.342636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:37.343633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:37.344657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:37.345850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 320/638] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:37.346874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa98a-011d-4757-9695-1704d9723fa2 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.347149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa98a-011d-4757-9695-1704d9723fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.347416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa98a-011d-4757-9695-1704d9723fa2 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.349923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 321/639] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:37.351982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa98a-011d-4757-9695-1704d9723fa2 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.387698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.434401 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd7b43-c0c4-416f-987c-2526ecd19950 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.455291 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.463718 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa98a-011d-4757-9695-1704d9723fa2 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.470805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.471615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.471896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.472164 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.485193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.487162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.500432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf807394-42b7-4251-a00c-a20ba02016fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.501225 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807394-42b7-4251-a00c-a20ba02016fd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.501525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807394-42b7-4251-a00c-a20ba02016fd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.501792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807394-42b7-4251-a00c-a20ba02016fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:37.508985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:37.509904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:37.510813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:37.510813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:37.510813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:37.510813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:37.511048 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 319/640] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:37.511793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.513578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf807394-42b7-4251-a00c-a20ba02016fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cf807394-42b7-4251-a00c-a20ba02016fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:37.523237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:37.524251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:37.525151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:37.525833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 322/641] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:37.526705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.526978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.527242 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.544402 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:37.545607 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:37.621407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 323/642] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:37.621601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18ca59-6a6c-4a79-905b-2aab025e68ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.621685 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.638630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.657563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.658481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.658912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.659310 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.660289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:37.660599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.662045 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.680104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 320/643] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:37.681514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.710216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.726436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b302277-2c9e-4353-ad73-b83749ee7204 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.764813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.765516 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.765830 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.766104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.766831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:37.767135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.768532 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.781197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 324/644] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:37.782065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.795736 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dc27d-bd73-4823-a496-769e6dd8cbf9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.834575 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.835606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.835887 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.836145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.836886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:37.837186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:37.839354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:37.854327 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 321/645] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:37.854327 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.886711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c9890-ae7e-4fee-a4a8-2f622cf37d65 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:37.901537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.924212 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:37.925030 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:37.925326 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:37.925595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:37.939596 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:37.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:37.948340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:37.949387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 325/646] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:37.950126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:37.959345 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:37.974462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26ba9-8f72-4402-8eae-0ee9a192592a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.009437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.010090 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.010604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.010604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.054132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c867d6-d65d-4cff-89ad-4a5d7a3aeff2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.079243 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.080718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.089842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.090963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.091390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.091870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.099174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.100902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:38.116674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:38.117662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:38.123512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 326/647] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:38.124055 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.124130 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.190247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 322/648] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:38.192599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.315973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff88dbc-dd6e-4d74-babb-09d2a104a5ee None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.330339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.356628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.357332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.357647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.357871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.358606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:38.358904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.360301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.365566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.368066 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f7150-053a-4bb9-9e25-2081d4ef2ba5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.369162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 323/649] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:38.370235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.409916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.410720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.410993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.411267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.421885 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.423348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:38.441307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:38.442261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:38.443192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:38.443192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:38.443192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:38.443192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:38.443192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 327/650] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:38.444540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.476939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f15d8-c14d-42dc-b443-d88289fa1288 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.509641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.510295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.510570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.510827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.511617 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:38.511918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.513369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.525193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 324/651] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:38.526065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.549505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2933d72-c9b7-4f2e-babd-036c9684c290 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.566650 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.583248 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.583774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.584759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.585081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.585387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.595579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.597031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:38.614526 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:38.615431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:38.616361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:38.616361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:38.616361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:38.616361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:38.616361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 328/652] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:38.616649 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.627225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72446512-49c5-45dc-921a-4eb5eb43c203 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.662875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.663750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.664024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.664288 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.674164 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.675587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6031601a-2113-4fd0-ad08-d4e91476d615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:38.692729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:38.693815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:38.694941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:38.694941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:38.694941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:38.694941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:38.694941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 325/653] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:38.695474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24949421-ac3c-4730-9864-5ece88fe7e1c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.695752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24949421-ac3c-4730-9864-5ece88fe7e1c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.696020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24949421-ac3c-4730-9864-5ece88fe7e1c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.700896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 326/654] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:38.700896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24949421-ac3c-4730-9864-5ece88fe7e1c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.721351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e510d6-6f6c-4414-bfb7-5407b2bb2aed None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.756700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.757409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.757678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.757940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.758660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:38.758963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.760351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.771738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 329/655] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:38.772518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.802207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.803444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24949421-ac3c-4730-9864-5ece88fe7e1c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.816307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:38.836879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.837571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.837841 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.838105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.838931 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=4528a20cb3b24aaab0ff2b52b56da1c6) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:38.839237 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.840663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.846576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 327/656] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/role_assignments?user.id=4528a20cb3b24aaab0ff2b52b56da1c6&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:38.847340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:38.881602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3857a7-216b-4d36-b2ad-8a493140b002 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.931029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:38.931902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.932253 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.932605 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.933471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:38.933850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:38.935333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59166071-f28e-405c-abad-ae2ca14d8e1f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:38.949319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 330/657] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:38.950696 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:38.951049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:38.951388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:38.969052 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520119-0795-4a9e-822e-1db69f051b3c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:38.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:38.970894 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:39.010725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.011578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.011978 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.012365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.013263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:39.013860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.015260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.034435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 328/658] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:39.035479 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.043005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 331/659] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:39.044107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.065542 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.079435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.151915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7250072-2e42-4586-a7c3-4bd90d19602a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.152645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434f55-e0cd-4236-9002-2db4c7d00e9e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.187865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.188756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.189101 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.189471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.190273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.191079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.191360 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.191610 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.196348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.197833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.199072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.200557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:39.211609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:39.212524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 332/660] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:39.218865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:39.219539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:39.220587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 329/661] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:39.223926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.301479 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.321030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.322791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef7db-a705-4a44-8229-c6d85c87a914 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.328771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c18d7-4878-4c65-8acf-76cc0bdf2ea3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.357648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.358433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.358710 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.358978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.366061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.366485 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.366864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.367140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.367413 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.367924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.378030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:39.381874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:39.382800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:21:39.383867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:39.385535 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 333/662] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:39.385703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:39.386612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:39.387466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:39.389124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.391582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 330/663] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:39.392506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.490617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2083f77-38ee-419f-80d9-0dab88f05880 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.503329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c02a5-1889-49a0-b791-f179cd902e67 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.525257 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.525939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.526216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.526487 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.527274 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:39.527582 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.529401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.540612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.543344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.544021 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.544294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.546993 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.548342 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:39.548858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.549390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 334/664] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:39.550316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.551481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.570985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 331/665] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:39.574596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.591009 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.665992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be926a8-c4c2-4ea4-8643-0106a1566da5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.681748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250bdf3-67eb-48ba-8999-a6d79fc3e395 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.700577 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.701271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.701568 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.701807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.702669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:39.702946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.704443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.718267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.718984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.719295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.719528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.720271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:39.720575 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.721959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.727412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 335/666] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:39.727412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.735335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 332/667] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:39.740425 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.786594 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.820734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:39.828537 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a6c83-11ce-4546-9299-3ed8ad12bdca None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.845939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9020e-121b-480a-8b22-9aa2e2c23f26 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:39.862701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.863441 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.863721 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.863993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.864820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=b4d24bc13b5c469cade02763acf072bd) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:39.865132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.866612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:39.872965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 336/668] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/role_assignments?user.id=b4d24bc13b5c469cade02763acf072bd&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:39.873794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:39.887548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:39.889225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:39.889785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:39.890254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:39.996736 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:39.999442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.004478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d85828-19be-4a7b-a3e2-fe591fc1d7d3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.030352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.045743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.046325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.046680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.047074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.120115 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.134269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.135742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.148097 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 333/669] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:40.150001 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.247026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 337/670] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:40.248230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.256821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045d6ff-1ec9-465a-934f-e4f72395071b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.272784 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.300102 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.301269 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.301732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.302151 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.313668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.315419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:40.336042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:40.337663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:40.338678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:40.338678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:40.338678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:40.338678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:40.340641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 334/671] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:40.341719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.373000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.375281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa8d14-6d8f-404d-a305-38aa3ac81103 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.421310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.422243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.422518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.422785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.436216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.437730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.451357 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b042ef-db02-4f77-901d-ad518ac3b8f0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:40.461153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:40.462151 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:40.463522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:40.463522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:40.463522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:40.463522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:40.463522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 338/672] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:40.463823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.501171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.502004 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.502282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.502555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.515434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.517168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:40.540059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:40.541033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:40.541997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:40.541997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:40.541997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:40.541997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:40.543124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 335/673] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:40.543752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.550348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.578099 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bd614-959c-440f-9078-c3a0786f0d9d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.626005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.626688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.626966 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.627251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.628215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:40.628519 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.630855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.645078 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.652561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 339/674] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:40.653525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.671589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5ec28-7037-4e18-8c16-2bf8380d6d96 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.715635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.716316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.716592 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.716863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.717623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:40.717934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.720181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.735591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 336/675] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:40.736499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.768835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b27326-0536-43cd-b055-c70bf79c90ff None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.805579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.806642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.807127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.807634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.808582 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:40.809089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.810682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.812620 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.834955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 340/676] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:40.834955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.861139 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a43f18-24fe-4860-877b-a298fd7142ec None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:40.905474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:40.906644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:40.906924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:40.907195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:40.917152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:40.922059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:40.924851 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:40.943323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:40.944535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:40.945677 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:40.952423 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 337/677] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:40.953676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:40.964911 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045cbde-1359-424a-b803-3c74e0717c28 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.011339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.012590 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.012936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.013329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.023464 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.025406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:41.042862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:41.043807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:41.044732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:41.047582 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 341/678] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:41.048719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.073597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dc39d-4a1d-44fe-bc64-0b2137c5be87 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.110104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.111218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.111753 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.112353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.118696 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.121223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.123155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:41.139871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:41.140976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:41.141929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:41.148870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 338/679] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:41.150094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.172514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a241ec1-0716-4590-9a28-f25921885332 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.208545 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.209230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.209536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.209808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.210647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:41.210956 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.212398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.218160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.238674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 342/680] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:41.238674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.259519 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f5a10-f961-45ee-a037-efa994994e0c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.294702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.295630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.295908 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.296180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.297003 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:41.297345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.298814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506f5e9-fc23-4e87-91de-37408cc2f19c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.310960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 339/681] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:41.312218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fdaee-6ba6-47bc-852d-4d3ee447ffa8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.312626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fdaee-6ba6-47bc-852d-4d3ee447ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.312902 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fdaee-6ba6-47bc-852d-4d3ee447ffa8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.314313 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 340/682] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:41.318398 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fdaee-6ba6-47bc-852d-4d3ee447ffa8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.356453 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf78044-765b-4805-aed4-1921e134caa8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.372154 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.393226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.393933 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.394211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.394478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.395257 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:41.395594 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.397033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.409067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 343/683] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:41.409880 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.433159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fdaee-6ba6-47bc-852d-4d3ee447ffa8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.454819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.469741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.470518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.470797 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.471062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.515067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d379-62d1-4ace-afd0-c21b10c3adb0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.541485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.543096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.549874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.550612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.551207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.551207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.551965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=de43fcf2258347a797867a32d8a7b1a3) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:41.552275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.553779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1175d8b-717c-4e96-819d-38c28047dd29 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.559768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 344/684] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/role_assignments?user.id=de43fcf2258347a797867a32d8a7b1a3&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:41.566749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.567065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.567314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.591648 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:41.593279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:41.639988 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.663385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 341/685] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:41.664730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.688731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.701794 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 345/686] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:41.703191 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.777488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21b695-6c28-4b40-93d3-7661b1f4e013 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.821170 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.821869 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.822150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.822429 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.823271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=d8507240870340478b50c27ae54381ef) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:41.823584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.825033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.831835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 342/687] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/role_assignments?user.id=d8507240870340478b50c27ae54381ef&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:41.832657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.837592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14968c6e-4828-4ab8-b342-bcde018ced6a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:41.893491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:41.895346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:41.895625 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:41.895896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:41.923253 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:41.924754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:41.957340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:41.958498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:41.959613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:41.959613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:41.959613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:41.959613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:41.959613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 346/688] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:41.966168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:41.974605 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:41.999117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9851-1530-4d85-a334-0493a8f44ff5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.005682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.076983 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.078058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.078438 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.078812 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.093411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.095153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:42.115703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:42.116877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:42.118011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:42.118011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:42.118011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:42.118011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:42.120095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 343/689] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:42.125442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.147915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e34ae-17ef-4801-82e7-6affa0dae917 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.193444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.194268 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.194700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.195005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.195997 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:42.196225 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.197762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.211393 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 347/690] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:42.218299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.247974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ae589-dc3c-4706-9b34-36a08f3700a3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.299694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.300843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.301306 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.301578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.302299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:42.302599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.304081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc046457-c167-4558-b469-aa7631775e1e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.306774 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.317803 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 344/691] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:42.319127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c645be6-72e6-48f5-8d20-a45fb5382a26 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.319405 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c645be6-72e6-48f5-8d20-a45fb5382a26 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.319674 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c645be6-72e6-48f5-8d20-a45fb5382a26 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.325201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.326494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 345/692] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:42.327368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c645be6-72e6-48f5-8d20-a45fb5382a26 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.354009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d0aa9-ce13-456f-a1b3-99ba9b44ce52 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.389267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.390138 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.390460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.390740 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.398434 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.399839 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-693e46cd-50d8-4dbf-8f45-3adc739080aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:42.416136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:42.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:42.418020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 348/693] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:42.421134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.421134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.421134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.437417 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:42.438871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:42.442329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c645be6-72e6-48f5-8d20-a45fb5382a26 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.482247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.482927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.483202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.483471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.484455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.486223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.508185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 346/694] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:21:42 2022] POST /identity/v3/users => generated 254 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:42.509415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.529390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 349/695] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:42.591800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.595333 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.681776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.684516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24682a3e-ba2e-4955-bf49-027944142a43 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.718201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b4df-0af2-4e84-bcef-274a4d4fb1cf None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.750548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192571de-1532-49d6-8792-606695016b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.751231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192571de-1532-49d6-8792-606695016b91 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.751500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192571de-1532-49d6-8792-606695016b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.751768 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192571de-1532-49d6-8792-606695016b91 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.752558 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192571de-1532-49d6-8792-606695016b91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:42.752868 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192571de-1532-49d6-8792-606695016b91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.754316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192571de-1532-49d6-8792-606695016b91 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.756083 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.757078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.757549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.757988 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.770267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:42.778478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:42.780033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:42.781194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:42.782020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 347/696] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:42.783693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 350/697] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:42.789893 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192571de-1532-49d6-8792-606695016b91 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.791336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:42.915055 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192571de-1532-49d6-8792-606695016b91 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.934800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.939821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f2069-5e49-4d35-b8c4-16688cae4d5a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:42.963972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.964932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.965219 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.965520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.966633 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=427c2d8e507749a4b94b7fe4ea37264d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:42.966955 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.969185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:42.971430 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:42.985300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 348/698] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:42 2022] GET /identity/v3/role_assignments?user.id=427c2d8e507749a4b94b7fe4ea37264d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:42.987804 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:42.988523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:42.988798 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:42.989073 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:42.990144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:42.990300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:42.991680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:43.002289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 351/699] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:43.007929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:43.127626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6544e5a-c45f-40ae-a850-0fca359b480a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:43.168073 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:43.168943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:43.169229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:43.169557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:43.181704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:43.183256 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:43.201272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:43.202395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:43.203566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:43.203566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:43.203566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:43.203566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:43.207114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 349/700] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:43.208501 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:43.317419 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020d2f7-3a9e-4a9f-9a2b-889556505196 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:43.352463 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:43.353136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:43.353445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:43.353729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:43.354452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:43.354759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:43.356146 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:43.373041 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 352/701] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:43.374445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:43.408213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:43.481898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e9c5d-7492-4ab5-85ec-859fabbd37f6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:43.518070 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:43.518886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:43.519180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:43.519436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:43.528414 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:43.530265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:43.547238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:43.549491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:43.551784 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 350/702] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:21:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:43.555325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:43.613614 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:43.672247 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5a4b3-450b-4194-856e-d4dade1d637b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:43.710544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:43.711612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:43.712091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:43.712561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:43.713659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:43.714166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:43.715825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:43.746771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 353/703] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:21:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:43.752202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:43.889673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:43.897897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6199f-f591-4629-bcc2-bd74c62a6827 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:43.941035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f9907-503e-4f97-95f1-a78108d3f13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:43.941035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f9907-503e-4f97-95f1-a78108d3f13e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:43.941035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f9907-503e-4f97-95f1-a78108d3f13e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:43.941035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f9907-503e-4f97-95f1-a78108d3f13e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.011189 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f9907-503e-4f97-95f1-a78108d3f13e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:44.012574 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f9907-503e-4f97-95f1-a78108d3f13e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:44.109888 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:44.113208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 351/704] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:44.122365 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.137151 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4e4c-7464-43f8-b8cd-02d040f9548e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.137516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4e4c-7464-43f8-b8cd-02d040f9548e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.137747 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4e4c-7464-43f8-b8cd-02d040f9548e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.139877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 352/705] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:44.157653 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.157934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.158209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.189268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:44.190465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:44.300681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c832-df8a-4ad6-8fe3-ea684f177472 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:44.311763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:44.339934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 353/706] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:21:44.347432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:44.348496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.348926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.349374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.366104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.377797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:44.379605 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:44.409298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:44.410476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:44.411601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:44.411601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:44.411601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:44.411601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:44.417260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 354/707] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:44.417260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.537725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b337a9-d48c-44a2-a93f-7ae098b63901 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:44.548022 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faada8-aa6d-4bbb-9c15-ceb59b453847 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:44.585666 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:44.589829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:44.590944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.591290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.591626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.592381 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:44.593985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:44.594117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:44.594699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.594982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.595255 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.595992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:44.596298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:44.597775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:44.614301 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 355/708] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:44.615527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:44.622270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:44.623233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:44.624132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 354/709] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:44.630177 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.638584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:44.730399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c685-4b5b-42a5-8161-9df7246949bb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:44.764739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:44.765670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.765954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.766235 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.773355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:44.774877 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:44.790113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:44.791178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:44.792129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:44.796555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 356/710] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:44.798175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.850783 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27275a9-e468-4172-958e-d53c3d0a0b8e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:44.899738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:44.900485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:44.900770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:44.901043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:44.901950 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:44.902254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:44.903818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:44.910154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 355/711] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:44.915114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:44.943438 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:44.983587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75079d8-c465-4d82-93f8-2775846e52e8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:45.009610 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:45.049816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:45.050529 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.050810 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.051084 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.051876 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:45.052708 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:45.054684 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5349f-cfa1-40bc-8c11-1e010731c9a1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:45.071189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 357/712] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:45.072503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1dc84-a67a-4f55-b88a-21e2bdfc95e0 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.072779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1dc84-a67a-4f55-b88a-21e2bdfc95e0 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.073052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1dc84-a67a-4f55-b88a-21e2bdfc95e0 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.077337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 358/713] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:45.078166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1dc84-a67a-4f55-b88a-21e2bdfc95e0 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:45.087591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb5dcc-87ce-4c2a-afb6-2090a91658fa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:45.137820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:45.138655 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.138961 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.139609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.140745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:45.141056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:45.143139 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b22951-6dd2-411c-9aca-949a0576497e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:45.151022 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 356/714] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:21:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:45.152491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5c7df-903d-4e39-b773-32fcdc241383 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.152777 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5c7df-903d-4e39-b773-32fcdc241383 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.153325 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5c7df-903d-4e39-b773-32fcdc241383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.177113 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fcc5c7df-903d-4e39-b773-32fcdc241383 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:45.178919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fcc5c7df-903d-4e39-b773-32fcdc241383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:45.192571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1dc84-a67a-4f55-b88a-21e2bdfc95e0 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:45.236752 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:45.237666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.237949 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.240366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.241345 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=b4d24bc13b5c469cade02763acf072bd) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:45.241667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:45.245215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:45.267361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 359/715] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:45 2022] GET /identity/v3/role_assignments?user.id=b4d24bc13b5c469cade02763acf072bd&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:45.299299 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 357/716] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:45.333284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:45.337360 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:45.452590 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2dc00a-3197-4eb1-8b12-6a0526324cfa None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.452775 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2dc00a-3197-4eb1-8b12-6a0526324cfa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.453151 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2dc00a-3197-4eb1-8b12-6a0526324cfa None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.455495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 358/717] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:45.460152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.460440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.460721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.467780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a69b3-df28-4a2d-9ded-9fa6215d2b7c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:45.485814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:45.487054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:45.515112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:45.516207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.516490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.516761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.517456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:45.518867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:45.549694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:45.562711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:45.563837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:45.564704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 360/718] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:45 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:45.575775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:45.583142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 359/719] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:45.621651 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:45.734719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7de1e-6eee-45fc-93ed-902161cd9cfa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:45.763040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:45.774800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:45.775772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.776145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.776513 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.788941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:45.797756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:45.798898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:45.799937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:45.802019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 360/720] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:45.809323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:45.833198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede9357-6501-4fa1-bc37-0d5421c0ba62 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:45.873545 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:45.874486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:45.875053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:45.876529 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:45.877411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] RBAC: Adding query filter params (name=network) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:45.877581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:45.878018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:45.880237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:45.894488 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 361/721] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:21:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:45.901894 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.003339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a6d45-cbe6-4f7e-a0e2-6c32e164ed14 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.039967 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.041219 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.041752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.042330 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.043730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:46.044474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:46.046480 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bfe6f-23b0-4049-85f4-848267cc9538 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:46.058117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 361/722] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:46.060444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b042349-e71a-453f-93fc-fdc6e17c5cd8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.060913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b042349-e71a-453f-93fc-fdc6e17c5cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.061402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b042349-e71a-453f-93fc-fdc6e17c5cd8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.062741 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd6436-9aaa-4380-afd0-b270d04c777b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.063358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 362/723] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:46.064456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b042349-e71a-453f-93fc-fdc6e17c5cd8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.095725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:46.102920 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.103957 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.104241 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.104522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.105352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] RBAC: Adding query filter params (type=network) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:46.105659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:46.108623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc21ca-e831-48cd-ac14-0a2a68d51d7f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:46.121768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 362/724] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:21:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:46.125736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.126343 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.127133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.161074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:46.164444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:46.235192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:46.254359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b042349-e71a-453f-93fc-fdc6e17c5cd8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.296624 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.297924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.298428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.298997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.312387 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:46.314510 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:46.316069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 363/725] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:46.333211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:46.334215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:46.335124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:46.335124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:46.335124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:46.335124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:46.337475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 363/726] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:46.346110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.350263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.478453 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd03f1-ce75-42f3-9b1d-1cdfd553284e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.483457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9439bc-614a-48f2-a411-b0f98ab59015 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.513671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:46.519496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.520164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.524175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.524466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.525202 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:46.525534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:46.528621 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:46.552456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 364/727] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:46.555314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.556230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.556567 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.556842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.559513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.584387 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:46.598785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:46.599746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:46.600606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 364/728] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:46.606962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.617994 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:46.705858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52634ee-9ef8-46ff-8b90-4a94cbc5ed84 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.744982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.745861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.746139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.746442 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.755733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:46.757314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:46.776302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:46.777967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:46.778943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 365/729] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:46.783736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.850436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fba18-3a02-468e-9020-fdff1762ffae None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:46.876517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:46.888641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:46.889350 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:46.890062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:46.890369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:46.891125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:46.891428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:46.892952 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:46.902939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 365/730] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:46.913431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:46.975107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e3192-b6b0-4aeb-88c2-16017da3a9f8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.019241 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:47.019930 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.020204 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.020476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.021355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:47.021666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:47.023102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:47.060836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 366/731] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:47.061923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:47.066426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:47.070534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b3ada-bf4e-4794-9a2b-7ad2876f557c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.131034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:47.134492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.134997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.135468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.157562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:47.159568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:47.188175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:47.189145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:47.190116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:47.190116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:47.190116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:47.190116 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:47.198679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 366/732] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:47.210281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:47.236385 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb4a30-07d4-4d57-96a4-ed8935edbbc0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.238784 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:47.285915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:47.286570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.286850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.287116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.393819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68f24-057f-4e85-8467-ab71c387b13d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.412669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:47.414150 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:47.424790 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:47.465991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:47.466685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.466965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.467297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.468040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:47.468343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:47.469862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:47.494628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 367/733] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:47.504084 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:47.550379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 367/734] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:21:47.556728 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:47.585668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:47.714776 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eb23f-e83a-42c9-87f3-f7fd95b82ac7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.761578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24009d06-9d32-4e47-adad-e25e7858faa2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.782370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:47.783189 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.783471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.783744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.807803 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:47.812931 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:47.824171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:47.824983 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.825279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.825559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6650a2ff-76c6-463e-b31d-098b6cfe0c9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:47.831806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:47.832770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:47.833813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 368/735] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:47.837259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384507bd-5985-4952-a5c9-6e0d1bbe2abc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.837259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384507bd-5985-4952-a5c9-6e0d1bbe2abc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.837259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384507bd-5985-4952-a5c9-6e0d1bbe2abc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.840709 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:47.841014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 369/736] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:47.842279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384507bd-5985-4952-a5c9-6e0d1bbe2abc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:47.842608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-13a5c159-469f-4d62-8ed9-5387f9f8de2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:47.862224 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:47.864597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 368/737] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0c4f3-146b-4ac9-9322-e030847d5889 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.867520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0c4f3-146b-4ac9-9322-e030847d5889 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.869901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0c4f3-146b-4ac9-9322-e030847d5889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.886379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-00d0c4f3-146b-4ac9-9322-e030847d5889 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:47.886696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:47.888042 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-00d0c4f3-146b-4ac9-9322-e030847d5889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:47.945588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:47.958433 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384507bd-5985-4952-a5c9-6e0d1bbe2abc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:47.969329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 369/738] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:47.969329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d726a7-3a8f-4e29-8f5e-13506edf6a38 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:47.969329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d726a7-3a8f-4e29-8f5e-13506edf6a38 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:47.969329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d726a7-3a8f-4e29-8f5e-13506edf6a38 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:47.969329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 370/739] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:47.969329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d726a7-3a8f-4e29-8f5e-13506edf6a38 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.000997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.001711 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.001991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.002301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.003103 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:48.003410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.004847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460124f9-52bf-41a6-9175-3f1de39bd998 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.023014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 370/740] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:21:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:48.026601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.026884 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.027155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.043512 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:48.044700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:48.078068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d726a7-3a8f-4e29-8f5e-13506edf6a38 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.112508 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.113176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.113544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.113770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.114492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:48.114795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.125432 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 371/741] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:48.125592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.125684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.142944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 371/742] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:48.143802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.151951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.193746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.231520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af168-0242-4207-91a4-ea1a0ffea033 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.250138 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186d51-15d7-4a97-91aa-911c94d54d12 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.271545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.272238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.272518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.272795 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.273552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.274935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.285460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.286120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.286404 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.286678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.292201 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 372/743] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:21:48 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:48.293224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.386134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.387630 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.417221 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.443604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.455025 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c5286-328c-43e2-8b68-fb6895131599 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.518098 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 372/744] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:21:48.519279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.624405 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.625285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.625561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.625870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.633169 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.634684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.639621 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d1a6-30f4-47a1-ac65-585a81ade724 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:48.649445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:48.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 373/745] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:48.654584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.689948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.691268 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.691694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.693009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.707707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:48.718861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:48.720027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:48.721075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:48.724408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 373/746] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:48.730211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.740832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.765635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faeb26-64c5-4006-b5f7-2f1858c4d0c6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.800839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.802039 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.802039 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.802263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.812685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.814168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.816335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-99bdd693-549b-4f5f-8ae0-d4a53d719b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:48.833278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:48.834205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:48.835114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:48.835114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:48.835114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:48.835114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:48.835114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 374/747] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:48.835937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb24b2-9b75-40b7-a60a-04a0e1ccdef6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.836040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb24b2-9b75-40b7-a60a-04a0e1ccdef6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.836295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb24b2-9b75-40b7-a60a-04a0e1ccdef6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.837816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 375/748] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:48.842079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb24b2-9b75-40b7-a60a-04a0e1ccdef6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.843054 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e35e3-00b9-4a14-8872-dea1a6ae656e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.878325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.879086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.879370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:48.879644 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:48.880428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:48.880737 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:48.882225 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:48.897779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 374/749] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:48.899099 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:48.960900 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb24b2-9b75-40b7-a60a-04a0e1ccdef6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:48.972749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:48.998915 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:48.999595 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:48.999935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.000211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.000963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:49.001294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.002684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8ab59-8b94-4175-a750-3d4522cd0478 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.010919 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdf550-8b67-4705-886d-d83fd7f65261 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.011536 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 376/750] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:49.012849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.013132 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.013431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.029335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:49.030651 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:49.052809 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.053886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.054315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.054792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.055850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:49.056342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.058061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.074899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 375/751] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:49.075969 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.081534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:49.098701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 377/752] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:21:49.099550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.190660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c51e4-e986-4fca-bf96-1c8cb2847ed9 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.212891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fac9a-c395-4e38-9d96-39c73f916da9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.215910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:49.225264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.225936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.226219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.226494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.227350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=3124c05981b6416ca04be4e734434d0b) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:49.227651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.229110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.235114 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 376/753] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/role_assignments?user.id=3124c05981b6416ca04be4e734434d0b&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:49.235912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.262751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.264017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.264477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.265013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.277790 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.280178 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:49.300270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:49.301477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:49.302458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:49.302458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:49.302458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:49.302458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:49.302458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 378/754] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:49.302751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.348687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:49.377741 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adac80-d8a3-420e-9c00-28ea4772f17d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.418800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef2f8-bc61-4faf-950f-7a4549f90e75 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.424323 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.425468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.425932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.426450 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.437421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.439902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.457136 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.457865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.458146 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.458431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.459156 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:49.460314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:49.461194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:49.462588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:49.463034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.463106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 377/755] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:49.463106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.486060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 379/756] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:21:49 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:49.486927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.563520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:49.661129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da858edd-2294-4bff-b670-def412ff9afc None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.685211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:49.690736 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e685177-e21d-448c-8ff2-4bc7e8026f8f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.725266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.725950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.726231 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.726506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.728874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:49.729198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.730639 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.733274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.734546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.734830 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.735105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.735907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:49.736218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.737700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.748352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 378/757] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:49.749725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.751723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 380/758] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:49.757821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.868014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb04af0-e0d5-4b8b-b548-16bfba3d2f2c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.882642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:49.887782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3ca55-3a2e-4ec2-bdd7-4a5ca735512f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:49.914761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.915807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.916238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.916658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.927887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.929627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.937675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:49.938453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:49.938833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:49.939205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:49.940174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=427c2d8e507749a4b94b7fe4ea37264d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:49.940582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:49.942149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:49.946416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:49.947366 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:49.948396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:49.948957 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 381/759] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/role_assignments?user.id=427c2d8e507749a4b94b7fe4ea37264d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:49.950786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 379/760] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:49.961446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:49.997276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:50.111351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c64c90-b462-4391-9de9-0a5647cda2ce None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:50.158958 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:50.160032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:50.160518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:50.160995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:50.162088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:50.162618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:50.164256 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:50.184493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 382/761] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:21:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:50.190435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:50.328405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44d417-db0a-473e-9822-026029d4365f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:50.377960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:50.378782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:50.379062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:50.379335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:50.391190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:50.392663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:50.415383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:50.416353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:50.417316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:50.417316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:50.417316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:50.417316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:50.417316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 380/762] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:50.423608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:50.561859 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:50.596374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835780e7-5f3d-4190-98e9-57aaaab57592 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:50.646018 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:50.646359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:50.646816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:50.646816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:50.649269 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:50.649269 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:50.649514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeff5e8-8315-4cc7-94f1-91b4187631ce None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:50.673190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 383/763] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:50.678367 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:50.717851 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:50.802927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a10648-d077-478c-8ba6-dc40b692c4c9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:50.803282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a10648-d077-478c-8ba6-dc40b692c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:50.803393 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cc94a-cc32-4c48-864e-73a53be59f51 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:50.803482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a10648-d077-478c-8ba6-dc40b692c4c9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:50.805147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 384/764] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:50.809989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:50.810263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:50.810538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:50.842422 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:50.843587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:50.846744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:50.847557 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:50.847837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:50.848110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:50.858399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:50.859946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:50.878616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:50.880021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:50.880940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:50.886704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 381/765] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:50.890185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:50.986042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 385/766] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:21:51.034468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:51.056382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:51.060328 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc211dbf-f244-4cbc-a0a4-6257cfdc8c2e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.120085 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.120757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.121036 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.121325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.122174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:51.122477 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.124129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.146193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 382/767] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:51.153032 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:51.157620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:51.212092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05fa3-83cb-4f54-93ce-9e672a5cb3b1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.262104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.262939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.263209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.263475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.264133 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.265694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:51.278614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:51.279935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:51.280772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 386/768] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:21:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:51.286825 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:51.382261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5edac-a26c-4409-95a3-e397bb26285d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.403088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca8cd-7755-4467-a593-7a70fefcf366 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.435656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:51.443388 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.444252 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.444668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.444943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.446611 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:51.446942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.448337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.454261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 387/769] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:21:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:51.459816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:51.469987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.470975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.471259 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.471576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/b4d24bc13b5c469cade02763acf072bd/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.478205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=b4d24bc13b5c469cade02763acf072bd, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.479676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.516060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 383/770] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:51 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/b4d24bc13b5c469cade02763acf072bd/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:21:51.571072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591225-da80-48de-8d74-f3e5e91584ee None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.607653 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.608354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.608628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.608909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.609733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:51.610040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.611433 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.617082 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 388/771] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:21:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:51.622864 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:51.636327 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:51.750909 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e830-d93e-49e4-a4f9-2366849f197d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.803689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.804796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.805339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.805807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.806858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] RBAC: Adding query filter params (interface=public, service_id=95f2198314ad4cbbaf8f03fb7765b0d1, region_id=RegionOne) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:51.807394 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.809041 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd99dd-5336-4956-a20d-c5916c825000 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.818643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 384/772] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:21:51 2022] GET /identity/v3/endpoints?service_id=95f2198314ad4cbbaf8f03fb7765b0d1&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:51.826263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:51.896812 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:51.945570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337d25-b5bb-4e25-b774-098a550b6ab4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:51.980541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4ed1a-5810-4225-990c-161656270386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:51.981227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4ed1a-5810-4225-990c-161656270386 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:51.981537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4ed1a-5810-4225-990c-161656270386 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:51.981811 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4ed1a-5810-4225-990c-161656270386 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:51.982555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4ed1a-5810-4225-990c-161656270386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:51.984029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4ed1a-5810-4225-990c-161656270386 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:51.991071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 389/773] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:21:51 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:52.085214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8a89-86ab-4b81-8563-db6a59536952 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.085765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8a89-86ab-4b81-8563-db6a59536952 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.085765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8a89-86ab-4b81-8563-db6a59536952 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.087493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 385/774] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:52.093144 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9280905-050a-46ec-b5de-ba61fe5e06a1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.093894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9280905-050a-46ec-b5de-ba61fe5e06a1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.094173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9280905-050a-46ec-b5de-ba61fe5e06a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.117562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d9280905-050a-46ec-b5de-ba61fe5e06a1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:52.118788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d9280905-050a-46ec-b5de-ba61fe5e06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:52.215022 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 390/775] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:52.249713 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8a89-86ab-4b81-8563-db6a59536952 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:52.336682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:52.342023 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfb6ca-6146-4e9f-8a8f-45682e78c8e7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.342579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfb6ca-6146-4e9f-8a8f-45682e78c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.343102 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfb6ca-6146-4e9f-8a8f-45682e78c8e7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.344991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 391/776] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:21:52.351363 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.351719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.352056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:52.376041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:52.380295 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8a89-86ab-4b81-8563-db6a59536952 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:52.435440 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:52.436275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.436541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.436815 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.442355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:52.453609 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:52.466240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:52.467237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:52.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:52.476316 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 386/777] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:52.481229 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:52.482126 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 392/778] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:52.515459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:52.612583 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d789482-0f71-4cc8-aafe-8847a12766be None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:52.637827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca18ec7-2e48-4069-993c-a3a625d933e7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:52.638683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:52.648226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:52.648890 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.649169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.649480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.650294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:52.650613 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:52.652299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3accdfd-ddd4-4e15-8404-aa9f7a291ee6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:52.666277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 387/779] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:52.667705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b089b-4e7e-4f18-8a6e-2b67ac4fa7d2 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.668005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b089b-4e7e-4f18-8a6e-2b67ac4fa7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.668285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b089b-4e7e-4f18-8a6e-2b67ac4fa7d2 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.669812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 388/780] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:52.670511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b089b-4e7e-4f18-8a6e-2b67ac4fa7d2 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:52.688509 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:52.689736 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.690283 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.690564 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.691234 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:52.693087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ea2a2688-4471-4dce-8475-eb869ca092fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:52.708247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:52.709280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:52.711321 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 393/781] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:52.712133 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.712133 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.712133 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.733547 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:52.734700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:52.781702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:52.782245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b089b-4e7e-4f18-8a6e-2b67ac4fa7d2 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:52.810785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 394/782] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:52.811733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:52.834552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:52.835458 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.835805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.836165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.852753 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:52.854429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:52.877033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:52.878703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:52.880379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:52.880379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:52.880379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:52.880379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:52.880379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 389/783] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:52.880379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:52.923981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eda3ba-d4a5-4aa3-a140-751bc6c83717 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:52.937203 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:52.971475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:52.972152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:52.972439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:52.972711 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:52.973450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:52.973754 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:52.975138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:52.984980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 395/784] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:52.985766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.024300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692eb9d-9b89-48e1-bc25-fcab942243f8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.029845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.076189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.077511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.078228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.079001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.094766 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:53.101142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bbafd-9a6b-48de-83e2-dec3be829aa4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:53.102509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:53.103468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:53.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:53.108858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 390/785] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:53.109906 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.151583 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.153172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.153687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.155041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.156451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:53.156973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.159197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.176575 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 396/786] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.177984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.212671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.216789 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577716a-6e7c-4aa4-883d-f822ef839f16 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.258615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.259607 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.260121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.260716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.261785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] RBAC: Adding query filter params (type=network) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:53.262300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.263888 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.269729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 391/787] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.270808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.294970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.297973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2a23-b451-453f-91c7-671244760ecb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.332708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.333469 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.333750 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.334021 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.334773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:53.335112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.336441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.349693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 397/788] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.350452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.381932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e341-608b-431e-8da3-94f123f725dd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.418736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.419989 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.420486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.420970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.430743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.432775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:53.450102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:53.451141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:21:53.452219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:53.455455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4f47-90c3-47ef-af9f-645d7ec829d7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.456067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 392/789] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:53.457166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.485217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.492655 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.493506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.494047 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.494349 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.495259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] RBAC: Adding query filter params (interface=public, service_id=fbefbfc4ea31418c8da981cfe60ea4dc, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:53.495674 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.497141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.510060 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 398/790] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/endpoints?service_id=fbefbfc4ea31418c8da981cfe60ea4dc&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.510938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.540261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.571855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fe8c0-d54b-4c42-bcba-993dfe877b72 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.609023 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.610275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.610735 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.611242 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.616007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7e49c-4138-48f0-b518-02c91e562c15 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.623285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.625196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:53.644106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:53.645110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:53.646074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:53.646074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:53.646074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:53.646074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:53.648666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 393/791] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:53.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.656645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.657442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.657864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.658142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.659098 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:53.659404 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.660894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.679829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 399/792] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.680539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.735436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.763032 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724229c5-75c1-430e-adeb-f6900a2dd759 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.778596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:53.801522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6428347-3efd-480c-bd1d-7ba8fe783685 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:53.802031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.802754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.803038 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.803307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.803991 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.805908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.817760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 394/793] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.818751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:53.843334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:53.844463 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:53.844960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:53.845487 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:53.846445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:53.847562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:53.849585 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:53.865187 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 400/794] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:53.874747 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.051325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314f82f-08d3-44c7-aea6-cb40686bd829 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.082800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed0571-161a-4384-92a6-b11a09653251 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.103277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.104900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.105424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.105931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.118187 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.120466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.122730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.122862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.122959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.122959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.123857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.132935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a933312f-500c-44d0-8b39-7276ab19a990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:54.143406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:54.144515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:54.145691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:54.145691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:54.145691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:54.145691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:54.145691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 395/795] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:54.150046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572720fe-983e-47ab-8a3d-94fa6193a3e0 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.150336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572720fe-983e-47ab-8a3d-94fa6193a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.150617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572720fe-983e-47ab-8a3d-94fa6193a3e0 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.151921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 396/796] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:54.152633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572720fe-983e-47ab-8a3d-94fa6193a3e0 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4951ca46-5873-40af-bda8-125f51a4c5b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:54.154446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:54.155563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:21:54.156599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:54.157510 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 401/797] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:54.162698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.162698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.162698 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.168374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.180915 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:54.182604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:54.266697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 402/798] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:54.267734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.286040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572720fe-983e-47ab-8a3d-94fa6193a3e0 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.334285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.334959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.335336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.335610 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.336391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:54.336692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.338348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.356045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 397/799] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:54.356941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.393399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa2bfc-d7d1-41c5-9fe3-25cc60633349 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.418413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.450019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.450719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.451460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.451745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.453072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:54.453416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.455173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.458112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.480199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 403/800] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:54.481081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.481571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968aff77-951f-4be4-b1f7-9f44c04cd2f5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.522482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.523733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.524162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.524652 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.536540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:54.545188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:54.546334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:54.547310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:54.554397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 398/801] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:54.555561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.607593 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e02d-fce0-4cbf-bfe8-37bf5118bb0b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.647696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.648579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.648829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.649118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.657148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.658752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.666152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf6e6-031f-43bb-88e8-90cade0c4931 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:54.673324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:54.674406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:54.675526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:54.679108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 404/802] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:54.680055 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.682235 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.704808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.705618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.705926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.706170 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.717319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.718818 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-99d89000-fb0c-4383-8d59-24f235bf7f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:54.738662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:54.739581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:54.740498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:54.740498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:54.740498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:54.740498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:54.742619 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.744472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 399/803] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:54.745686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb788a-a24e-4167-99e8-4d2d5d532680 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.746038 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb788a-a24e-4167-99e8-4d2d5d532680 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.746348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb788a-a24e-4167-99e8-4d2d5d532680 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.747805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 400/804] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:54.748491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb788a-a24e-4167-99e8-4d2d5d532680 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.796572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4d3f5-d8ba-4d29-a4d2-c1c57d1addaa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.831838 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.832823 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.833177 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.833476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.834225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:54.834534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.835908 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.849311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 405/805] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:54.849311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:54.857222 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb788a-a24e-4167-99e8-4d2d5d532680 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.890566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:54.891154 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.891430 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.891704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.892484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:54.892785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:54.894258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5dbaa-79df-448d-8a66-64bb38a3e55c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:54.907150 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 401/806] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:54.908175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:54.908454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:54.908723 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:54.926103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:54.927281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:54.945059 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.967968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12d88-3184-4fa9-abab-53152cebed14 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:54.982117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:54.996721 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 402/807] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:54.997599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.012755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.013454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.013730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.013996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.014718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:55.015019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.016425 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.028668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 406/808] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:55.029819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.118275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c38c-9f48-44e3-be70-2695bd5b4927 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.132923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a51fb3-069d-43fa-85bb-ca3ac7d079a7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.154979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.156538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.157007 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.157503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.169268 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.169487 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.170410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.170720 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.171067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/3124c05981b6416ca04be4e734434d0b/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.173745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.187249 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=3124c05981b6416ca04be4e734434d0b, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.188743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:55.195052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:55.196226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:55.197155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:55.197155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:55.197155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:55.197155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:55.197155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 403/809] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:55.197484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.208432 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 407/810] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:55 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/3124c05981b6416ca04be4e734434d0b/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:55.209105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.217876 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:55.245119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:55.339681 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac694ff-a0c2-4cee-bf8b-3ec58e08c249 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.348385 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a98b-e6cf-4a39-ab48-580808dea2b8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.387779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.388722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.389074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.389454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.390317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.390519 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.391525 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.391948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.392074 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.392465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.400511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.402249 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:55.407915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:55.409072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:21:55.410184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:55.412379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 404/811] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:55.413492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:55.422459 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:55.423541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:55.424657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 408/812] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:55.425292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.497144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:55.533842 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0e62a-8c92-42e7-9848-33f15050147b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.538533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:55.559211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc6d6c-b80b-40e3-b4f6-2a75d726622b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.572436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.573153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.573469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.573775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.574612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:55.574919 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.576308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.593691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 405/813] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:55.594712 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.602468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.603379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.604307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.604742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.605622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:55.606506 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.608720 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.615704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 409/814] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:55.621845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.711199 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e61a7-97a3-449a-b2bf-9f67ecd0dafa None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.741993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:55.750143 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.750821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.751233 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.751464 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.752415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:55.752683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.754185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.755591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dac1b3-77c5-4f1a-9e4b-38ffe3435187 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.771873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 406/815] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:55.771873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.809854 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.811195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.812562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.812995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.822849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.825476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:55.845543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:55.846730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:21:55.847912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:55.850661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 410/816] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:55.851571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.864516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:55.891536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0440dec-7032-4011-8ed3-25aa9822eaa2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:55.937604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:55.938642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:55.939205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:55.939702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:55.940635 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:55.941149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:55.943031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:55.953605 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 407/817] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:55.954882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:55.978657 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d8fc-f31a-4077-bf2c-2a225ad056bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.021539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.032827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.033773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.034065 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.034316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/427c2d8e507749a4b94b7fe4ea37264d/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.036553 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=427c2d8e507749a4b94b7fe4ea37264d, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.038646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.061345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 411/818] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:21:55 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/427c2d8e507749a4b94b7fe4ea37264d/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:21:56.062743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:56.102700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5217caa-f73d-4f72-8c54-4ca11f631cf5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.182116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.184075 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.184359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.184633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.186104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:56.186429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.188014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d99ba-f706-4c33-9aa9-122108300c2a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.205205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.214764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 408/819] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:21:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:56.217699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea28c0-9ee5-41ca-b6d6-25d19ca6cba3 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.218284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea28c0-9ee5-41ca-b6d6-25d19ca6cba3 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.218766 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea28c0-9ee5-41ca-b6d6-25d19ca6cba3 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.225367 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 409/820] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:56.226395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea28c0-9ee5-41ca-b6d6-25d19ca6cba3 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:56.258105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ead98-0dba-4fea-88cd-ebbe3c93004c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.296904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5d180-91db-4b55-9327-5434ec6b3487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.297977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5d180-91db-4b55-9327-5434ec6b3487 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.298489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5d180-91db-4b55-9327-5434ec6b3487 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.299002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5d180-91db-4b55-9327-5434ec6b3487 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.299957 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5d180-91db-4b55-9327-5434ec6b3487 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.302415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5d180-91db-4b55-9327-5434ec6b3487 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.327217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 412/821] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:21:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:56.329041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.329586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.330193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.359634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea28c0-9ee5-41ca-b6d6-25d19ca6cba3 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.360372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:56.362093 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:56.370363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.398299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.398965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.399248 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.399524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.477545 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.479193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.492538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 413/822] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:56.517648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:56.597270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 410/823] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:21:56.603508 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:56.623812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.629599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.709043 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f636ec-6d1e-452e-a8cd-317561743a91 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.731574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d275b-d7c5-430a-99eb-2c2569c9b46e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.744732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.745581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.745858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.746130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.746792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.748169 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:56.760844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:56.761889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:56.762948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 414/824] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:56 2022] GET /identity/v3/services/network => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:56.768811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.769565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:56.770039 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.770322 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.770602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.786511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.788335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:56.807521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:56.808462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:56.809408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:56.809408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:56.809408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:56.809408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:56.811260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 411/825] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:56.818247 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:56.885472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.887846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee9e7c-a9bd-44a0-a1a0-17f9319d191a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.925555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:56.949589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:56.950275 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:56.950553 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:56.950827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:56.951546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:56.951853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:56.953346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:56.957745 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834689a3-88f3-423b-b03a-9832cfcd68bd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:56.959106 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 415/826] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:21:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:56.966213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:57.037370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:57.038435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.038940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.039413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.040392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:57.040910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:57.042534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:57.065225 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 412/827] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:57.070658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:57.151377 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1d384-f617-4be9-9ae3-16379b95ba3b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:57.186908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59fc56-134c-4e91-9300-7e1ba7c4baf5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:57.210382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:57.226773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:57.227966 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.228454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.228928 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.237016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:57.238756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:57.245924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:57.246608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.246885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.247158 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.247872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:57.248175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:57.249699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:21:57.255873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:57.256764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:21:57.257796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:57.261884 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 413/828] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:57.268693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 416/829] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:21:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:57.269910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:57.274837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:57.384216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a6685-6832-4f33-b75c-463fba79d13f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:57.402629 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:57.430781 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:57.431467 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.431746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.432021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.432827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:57.433131 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:57.434617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:57.464903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 414/830] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:57.479335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:57.534208 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:57.536602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d01a9-ac0d-44a8-8619-b1d9fb771e61 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:57.597292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd0406-e07d-4ed3-8b53-d9c8eb2cd25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:57.598229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd0406-e07d-4ed3-8b53-d9c8eb2cd25c None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.598659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd0406-e07d-4ed3-8b53-d9c8eb2cd25c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.599082 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd0406-e07d-4ed3-8b53-d9c8eb2cd25c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.600017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd0406-e07d-4ed3-8b53-d9c8eb2cd25c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:57.601606 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd0406-e07d-4ed3-8b53-d9c8eb2cd25c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:57.625929 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83cf8-4247-4b13-a3bd-fd6303c9161b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:57.636165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 417/831] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:21:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:21:57.678827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:57.679887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.680318 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.680782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.681823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=b4d24bc13b5c469cade02763acf072bd) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:57.682294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:57.684056 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af221a90-2dfc-4002-96a5-30641fc97291 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:57.692181 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 415/832] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:21:57 2022] GET /identity/v3/role_assignments?user.id=b4d24bc13b5c469cade02763acf072bd&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:57.702941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52f17d-ce09-41fd-a556-ce3b3cfe7300 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.703285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52f17d-ce09-41fd-a556-ce3b3cfe7300 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.703565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52f17d-ce09-41fd-a556-ce3b3cfe7300 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.705310 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 418/833] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:57.711169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:57.711691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:57.712178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:57.738592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:57.740104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:57.817083 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 416/834] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:57.852392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52f17d-ce09-41fd-a556-ce3b3cfe7300 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:57.987296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52f17d-ce09-41fd-a556-ce3b3cfe7300 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:57.989048 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:58.036281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.037101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.037475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.037722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.049719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:58.057390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:58.058842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:58.060659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 419/835] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:58.064844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:58.103580 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:58.186937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da0a5b-abd7-4f74-baed-5ef21a0ced33 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:58.222544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.224038 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.224531 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.225009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.226013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:58.226549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:58.228136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:58.237350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 417/836] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:21:58.242464 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:58.345332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af0d3b-e2b6-4681-83ad-4c7b2d381552 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:58.380038 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.380841 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.381117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.381419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.392963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:58.394633 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:58.416880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:58.417861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:58.418801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:58.418801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:58.418801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:58.418801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:58.418801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 420/837] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:58.424271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:58.528276 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:58.537813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d51d7-f3e9-4504-8706-ab9953a8b55f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:58.572972 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.573681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.573962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.574237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.574967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:58.575273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:58.576658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda5e1-f606-4de2-933b-4afb95811699 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:58.590675 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 418/838] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:58.596476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:58.613818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:58.662925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f235f-e958-4cfe-ba70-d33c788f03d1 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.663562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f235f-e958-4cfe-ba70-d33c788f03d1 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.664046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f235f-e958-4cfe-ba70-d33c788f03d1 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.667147 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 419/839] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:58.671716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.672245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.672755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.694834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:58.696720 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:58.707388 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713c747-c7bb-4c7e-a237-be88ee338be7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:58.743486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.744319 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.744600 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.744871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.753397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:58.755066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:21:58.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:58.771817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:58.774005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 421/840] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:21:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:58.781545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:58.781725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 420/841] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:21:58.802027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:58.881013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:58.902983 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:58.913926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a72a5d-f110-458a-ab33-6c18dbac1879 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:58.934628 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f0e16-9001-47a1-bc7e-02c5a0f426fa None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:58.965408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.966869 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.967328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.967744 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.969369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:58.969831 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:58.971408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6acd3a-8891-4982-8dff-39ea93abbfa0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:58.973734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:58.975306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.975589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.975862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.986607 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:21:58.989620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 422/842] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:21:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:58.993287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2d192-c6fa-4d2a-8bfb-3c3f22bb9b1f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.993287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2d192-c6fa-4d2a-8bfb-3c3f22bb9b1f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.993287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2d192-c6fa-4d2a-8bfb-3c3f22bb9b1f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:58.993287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 423/843] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:21:58.994319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:58.995319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:21:58.996182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:58.998679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:58.998965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:58.999236 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.000407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 421/844] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:21:59.001277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.017449 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:21:59.018678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:21:59.089400 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 424/845] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:21:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:21:59.089400 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.122375 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890bab6-e53c-4936-95a6-4a6ee85c5b21 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.164010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.164529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.164791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.165147 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.165624 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.180769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.205059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeb50b-299f-4fa9-81e5-1408017297a1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.240192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.240863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.241130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.241428 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.242177 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:59.242478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.243844 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.244679 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.246663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.256655 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 425/846] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:59.256655 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.348603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 422/847] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:21:59.349430 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.366552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185ff9b-f63a-4ec6-b4c5-c2175dec1044 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.392442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.394178 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.407743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.408662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.408951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.409210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.409890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.411484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:59.423971 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:59.424965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:59.430446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 426/848] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:59.431390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.456676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1de6b-687a-4c0c-ab4e-27aaf06fa594 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.491621 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.492506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.492764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.493086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.503384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.504878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:59.523706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:59.524791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:59.525743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:59.525743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:59.525743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:59.525743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:21:59.525743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 423/849] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:59.525743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.534373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc363961-6590-4bff-bad4-5852b6f310fc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.570080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.570878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.571153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.571421 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.582887 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.584389 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:21:59.602820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:21:59.603760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:21:59.604764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:21:59.604764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:21:59.604764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:21:59.604764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:21:59.604764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 427/850] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:21:59.605058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.627623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.636064 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a07d71-251f-4abf-9461-9c80c8dfeb41 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.649645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.673453 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.674123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.674401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.674671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.675377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:59.675680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.677065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.681817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 424/851] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:59.687110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.712934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61473f7-ecb7-439a-8b73-b74f925caba8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.750545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.751466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.751899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.752198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.752889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:59.753206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.754599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.771889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 428/852] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:59.771889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.795742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249b312-2f53-4587-b269-19f3c66fdaba None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.833546 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.834537 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.834821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.835172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.835900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:59.836211 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.837617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.849538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 425/853] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:59.850346 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.869523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.873359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:21:59.876716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cce467-2f7f-40d2-a259-69945b9dfcba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:21:59.912852 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:21:59.913555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:21:59.913835 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:21:59.914108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:21:59.914821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:21:59.918427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:21:59.918427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:21:59.922646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 429/854] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:21:59.923511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:21:59.976245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c98237-7b55-42e7-9f92-c20b5797ebeb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.017098 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.018105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.018451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.018795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.028141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.030133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:00.052672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:00.053908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:22:00.054952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:00.058851 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 426/855] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:00.059848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.073504 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b021771-bff5-486e-9652-1ddab2bbba26 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.115346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.116710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.116710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.116916 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.124912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.126392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:00.141483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:00.142508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:00.143427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 430/856] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:00.149366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.174056 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.183887 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.202456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79025790-931d-4ba5-b0e2-6926d353628c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.237284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.237965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.238232 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.238500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.239527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:00.239643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.241195 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e7208-c2f3-4b49-b4ec-72f0ffe3b988 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.258604 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 427/857] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:00.263492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07070ec-8fff-4fca-964f-423c80713248 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.263782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07070ec-8fff-4fca-964f-423c80713248 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.264068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07070ec-8fff-4fca-964f-423c80713248 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.264429 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4b3d6-d2b7-4963-86e2-d3cba7775f14 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.265542 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 428/858] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:00.266152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07070ec-8fff-4fca-964f-423c80713248 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.303615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.304305 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.304563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.304834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.305658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:00.305964 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.307435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cffd1b-271f-44f8-a48f-530885da63c2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.324781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 431/859] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:00.326204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.326493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.326766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.343992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:00.345161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:00.373509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07070ec-8fff-4fca-964f-423c80713248 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.411667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.413161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.413762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.414308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.419067 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.421454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 432/860] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:00.430457 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.449845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.496923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.498880 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.540358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3beb-3b60-401b-bb31-cffecf4b2f4f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.575071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.575732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.576011 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.576282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.577118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=3124c05981b6416ca04be4e734434d0b) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:00.577483 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.578953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.585301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 433/861] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/role_assignments?user.id=3124c05981b6416ca04be4e734434d0b&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:00.585715 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.618539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 429/862] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:00.623952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.686302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.687636 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.727358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049336-3d4d-40d2-81af-93df8a524be0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.768137 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.769443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.769941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.770417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.771297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.771404 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433772dc-85ef-4648-8685-e0f09d870390 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.773008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:00.787095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:00.788173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:00.789049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:00.792022 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 434/863] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:00.798253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.813721 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.814529 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.814803 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.815069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.827122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.828654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:00.847435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:00.848411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:00.850996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:00.850996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:00.850996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:00.850996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:00.850996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 430/864] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:00.855186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.914808 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828493d-6c02-45ed-90d2-15f9c1d5d03f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.960943 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:00.961665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:00.961937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:00.962213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:00.962960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:00.963326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:00.964807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:00.967368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:00.971184 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 435/865] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:00.976833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:00.982277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9927971-17b9-4947-a143-d0ee7bb685fb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:00.987962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:01.025012 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:01.025748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:01.026028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:01.026302 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:01.027023 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:01.027325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:01.029493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:01.043868 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 431/866] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:01.049529 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:01.106597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330792f8-58a5-45e2-a86f-8b46c7d7dd5f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:01.148618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:01.149436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:01.149781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:01.150119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:01.150896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:01.151267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:01.152727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:01.158552 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 436/867] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:01.172351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd188007-1a24-471f-9fad-2701454905b8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:01.211820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:01.212564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:01.212839 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:01.213107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:01.221457 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:01.222965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:01.239190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:01.241341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:01.243443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 432/868] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:01.246758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:01.326450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:01.360748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83706793-cb4a-465d-a3ea-450c0a2ee86e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:01.395764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:01.396431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:01.396707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:01.396973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:01.397791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:01.398155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:01.399596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10946dfb-b6fa-4a50-b784-4beb1b41cb30 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:01.412584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 437/869] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:01.418220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:01.431215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:01.525564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6db79-9bec-4b49-91dd-088826553618 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:01.560163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:01.560820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:01.561098 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:01.561400 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:01.562414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=427c2d8e507749a4b94b7fe4ea37264d) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:01.562725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:01.564267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e7da-f085-49c0-908c-35be757d3362 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:01.570678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 433/870] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:01 2022] GET /identity/v3/role_assignments?user.id=427c2d8e507749a4b94b7fe4ea37264d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:02.228772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b854e8-e9fe-4b96-b200-10a0fde3a66e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:02.229325 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b854e8-e9fe-4b96-b200-10a0fde3a66e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:02.229325 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b854e8-e9fe-4b96-b200-10a0fde3a66e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:02.230917 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 438/871] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:02.236386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a074156-d2a7-4137-92e4-052b2b699a73 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:02.236916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a074156-d2a7-4137-92e4-052b2b699a73 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:02.237432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a074156-d2a7-4137-92e4-052b2b699a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:02.255198 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-1a074156-d2a7-4137-92e4-052b2b699a73 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:02.256693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-1a074156-d2a7-4137-92e4-052b2b699a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:02.327682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 434/872] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:02.350318 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b854e8-e9fe-4b96-b200-10a0fde3a66e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:02.458572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b854e8-e9fe-4b96-b200-10a0fde3a66e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:02.489144 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:02.495712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a911a5-b52f-4596-9915-b6377b8aa333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:02.496282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a911a5-b52f-4596-9915-b6377b8aa333 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:02.496733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a911a5-b52f-4596-9915-b6377b8aa333 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:02.497013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a911a5-b52f-4596-9915-b6377b8aa333 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:02.497787 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a911a5-b52f-4596-9915-b6377b8aa333 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:02.499486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a911a5-b52f-4596-9915-b6377b8aa333 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:02.515271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 439/873] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:02 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:22:03.048776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0a32d-ee05-43c0-95a4-85a26dab589e None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.049353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0a32d-ee05-43c0-95a4-85a26dab589e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.049353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0a32d-ee05-43c0-95a4-85a26dab589e None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.050856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 435/874] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:03.056093 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.056385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.056664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.075010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:03.076933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:03.159641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 440/875] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:03.178441 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0a32d-ee05-43c0-95a4-85a26dab589e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:03.181407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:03.289862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0a32d-ee05-43c0-95a4-85a26dab589e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:03.328594 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e685-0073-4480-a743-381feb835eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:03.329603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e685-0073-4480-a743-381feb835eca None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.329877 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e685-0073-4480-a743-381feb835eca None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.330144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e685-0073-4480-a743-381feb835eca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.342015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956e685-0073-4480-a743-381feb835eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d956e685-0073-4480-a743-381feb835eca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:03.350985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:03.353100 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:03.355024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 436/876] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:22:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:03.361262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:03.458392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7ab99-f352-4a39-a18d-84d5d8a4e16b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:03.475337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:03.494703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:03.496248 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.496524 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.496800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.497597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:03.497906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:03.499369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:03.508801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 441/877] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:22:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:03.509166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dc90d-126d-4b1a-be52-0e5b76d061f3 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.509462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dc90d-126d-4b1a-be52-0e5b76d061f3 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.509768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dc90d-126d-4b1a-be52-0e5b76d061f3 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.511261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 437/878] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:03.514134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:03.516213 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.516497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.516774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.534725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:03.536009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:03.596512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:03.613509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 438/879] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:03.633169 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:03.635052 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa61527-d78d-4c2c-a040-21845965ecb8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:03.682553 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:03.683853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.684315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.684773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.697526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:03.699297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:03.745431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:03.746437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:03.747411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:03.747411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:03.747411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:03.747411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:03.750931 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 442/880] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:03.756273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:03.790654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bf60e-c141-4d1e-a477-aa7667e98818 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:03.804810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:03.868878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486acde7-5c77-4822-ae57-71001a45f18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:03.873750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486acde7-5c77-4822-ae57-71001a45f18f None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.874130 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486acde7-5c77-4822-ae57-71001a45f18f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.876138 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486acde7-5c77-4822-ae57-71001a45f18f None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.877077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486acde7-5c77-4822-ae57-71001a45f18f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:03.882824 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929abd67-f858-42c2-b83f-e9b612c12e97 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:03.885092 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486acde7-5c77-4822-ae57-71001a45f18f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:03.928152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:03.929290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.929822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.930298 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.931517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 439/881] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:03 2022] POST /identity/v3/services => generated 269 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:22:03.932100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:03.932628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:03.934386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:03.936286 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a6a6-a7f0-459d-aed7-d5fc165c5da4 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.936602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a6a6-a7f0-459d-aed7-d5fc165c5da4 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.936932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a6a6-a7f0-459d-aed7-d5fc165c5da4 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.942220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 440/882] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:03.947654 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:03.947946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:03.948221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:03.960164 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 443/883] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:03.961631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:03.988790 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:03.992672 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:04.088941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c0af-3a96-4860-908a-b600843f2db0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.094049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:04.122537 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:04.136679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.137892 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.138613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.139322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.148977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 441/884] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:04.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.151155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:04.169378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:04.170569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:22:04.171718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:04.174155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 444/885] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:22:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:04.177547 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.181215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.306908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5642b8-fe37-427e-a8ba-50d6a115a66e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.321103 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba54f3-36d7-4738-a3e0-af6b3dbe026a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.349590 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.350825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.351208 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.351585 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.352503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.354727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:04.364849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.366153 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.366632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.367058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.368085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:04.368540 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:04.369775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:04.370808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:04.371669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:04.373423 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:04.376625 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 442/886] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:04.377634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.402041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 445/887] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:04.403764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.409617 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:04.445954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:04.516483 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2cefc-b57f-4c72-9ca8-4f67052b75da None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.540265 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9a138-c90b-4a8d-90bb-644451026453 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.560011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.560687 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.560970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.561270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.562119 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:04.562427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.563831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:04.573390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 443/888] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:04.576444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.583855 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9154695-3080-47a5-9c36-bcae0fb1c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.584518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9154695-3080-47a5-9c36-bcae0fb1c671 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.584885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9154695-3080-47a5-9c36-bcae0fb1c671 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.585151 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9154695-3080-47a5-9c36-bcae0fb1c671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.663949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9154695-3080-47a5-9c36-bcae0fb1c671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.665364 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9154695-3080-47a5-9c36-bcae0fb1c671 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:04.668907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:04.715292 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4548-13ff-4894-94c5-b12668d931de None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.726459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:04.752404 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.752773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.753017 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.753389 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.754218 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:04.754659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.755941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:04.769277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 444/889] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:04.801005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 446/890] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:04.807105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.813886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9b10-5b22-4e73-a1d0-7a267db26088 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.814219 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9b10-5b22-4e73-a1d0-7a267db26088 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.814435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9b10-5b22-4e73-a1d0-7a267db26088 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.816111 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 447/891] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:22:04.822566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c36e-6949-4107-8711-a0385554175e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.822919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c36e-6949-4107-8711-a0385554175e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.823192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c36e-6949-4107-8711-a0385554175e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.841682 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-71e2c36e-6949-4107-8711-a0385554175e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:04.843262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-71e2c36e-6949-4107-8711-a0385554175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:04.925941 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9be492-4232-4ace-a2e2-a2200d065ce0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:04.928396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 448/892] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:04.957147 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c36e-6949-4107-8711-a0385554175e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:04.975482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:04.976312 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:04.976593 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:04.976865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:04.993139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:04.995332 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.009071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:05.014773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:05.015837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:05.016763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:05.016763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:05.016763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:05.016763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:05.016763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 445/893] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:05.021975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.052204 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.088582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c36e-6949-4107-8711-a0385554175e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.125708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.126714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.127089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.127456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.128210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.129825 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.135186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813d74-79b3-4c30-831d-b6c2e1327b89 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:05.142998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:05.144112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:22:05.144981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:05.149611 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 449/894] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:22:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:05.150920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.177422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.178552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.178832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.179371 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.180170 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:05.180480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.182838 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.199618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 446/895] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:05.204633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.273745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.278729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800150ec-ba03-4f2a-95ad-2b8b7615c57c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.314200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.316718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.317758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.318235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.318696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.319714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:05.320277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.321906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.332751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79140e43-8559-439e-8d88-f43d63912df4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.333489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 450/896] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:05.334664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.374615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.375801 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.376269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.376830 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.384442 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.386536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:05.401008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:05.401940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:05.402856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:05.410389 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 447/897] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:05.411585 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.456752 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572ba6-2669-4b42-a437-805c738c4cb6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.492015 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.492686 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.492964 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.493230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.493970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:05.494275 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.495645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.506371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 451/898] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:05.508144 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.537663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.551617 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b87e1-7dbf-40db-b68d-0075036f60ea None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.604841 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.606205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.606731 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.607425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.608901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:05.609671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.611954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.628948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d912bfc-2173-4a7c-abaf-3e68171c60e8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.634229 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 448/899] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:05.636013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.647576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.676416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.677173 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.677833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.678112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.679224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] RBAC: Adding query filter params (interface=public, service_id=c13fe2ccd4f6404984a5038ae0093ad3, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:05.679549 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.681390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.692743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 452/900] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/endpoints?service_id=c13fe2ccd4f6404984a5038ae0093ad3&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:05.694149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:05.756072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a11ebd-9a8d-437b-89ce-037cd7b765ff None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.792294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.792955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.793230 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.793519 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.794165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:05.794343 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] RBAC: Adding query filter params (role.id=f8e128c40c034a9587c922be6a3fc432, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=3124c05981b6416ca04be4e734434d0b) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:05.794649 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.796122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e5824-e7e3-415d-8c81-d3965b241074 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.805285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 449/901] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/role_assignments?user.id=3124c05981b6416ca04be4e734434d0b&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=f8e128c40c034a9587c922be6a3fc432 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:05.812989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7801-5fc8-4699-bc9a-82b0df6303c0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:05.852565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4a540-aa37-48fd-a4ce-8e3c3120bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:05.854880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a540-aa37-48fd-a4ce-8e3c3120bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:05.855159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a540-aa37-48fd-a4ce-8e3c3120bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:05.855438 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a540-aa37-48fd-a4ce-8e3c3120bbe8 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:05.856293 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4a540-aa37-48fd-a4ce-8e3c3120bbe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:05.857878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4a540-aa37-48fd-a4ce-8e3c3120bbe8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:05.864876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 453/902] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:22:05 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:22:06.278031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa51ed-a012-4478-b20e-af2eb463fd50 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:06.278570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa51ed-a012-4478-b20e-af2eb463fd50 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:06.278570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa51ed-a012-4478-b20e-af2eb463fd50 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:06.280640 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 450/903] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:06.286279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:06.286568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:06.286842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:06.303813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:06.304990 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:06.374702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 454/904] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:06.400924 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa51ed-a012-4478-b20e-af2eb463fd50 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:06.474906 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:06.506993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa51ed-a012-4478-b20e-af2eb463fd50 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:06.542765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:06.543494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:06.543770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:06.544035 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:06.544683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:06.546113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:06.562767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:06.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:06.564657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 451/905] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:22:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:06.570581 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:06.658454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:06.676292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b49eb-1c06-4058-a75e-941819f6b850 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:06.715496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:06.716587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:06.717067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:06.717608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:06.718579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:06.719096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:06.720681 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:06.726223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 455/906] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:06.731603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:06.832060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496fc8b-79b1-44e8-a24a-f09d8bd41ccb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:06.866596 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:06.867273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:06.867631 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:06.868115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:06.869261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:06.869582 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:06.870965 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:06.873692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:06.882800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 452/907] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:06.885168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:07.002729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a6717-79f9-414e-b6ce-fdf981c0de78 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:07.038430 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:07.039095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:07.039368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:07.039636 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:07.040417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] RBAC: Adding query filter params (interface=public, service_id=3cbb5b60acad4801b8612db8c1dc3a82, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:07.040719 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:07.042166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9abbc9-ce11-4e4f-9ca2-44914e511399 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:07.047863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 456/908] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:22:06 2022] GET /identity/v3/endpoints?service_id=3cbb5b60acad4801b8612db8c1dc3a82&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:07.053301 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:07.097091 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff7e5-ced5-4054-acdb-e5f2d8812e13 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:07.097392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff7e5-ced5-4054-acdb-e5f2d8812e13 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:07.097612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff7e5-ced5-4054-acdb-e5f2d8812e13 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:07.099202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 457/909] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:22:07.104089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eefdee-5c13-47fc-9d96-240d0c117e41 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:07.104374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eefdee-5c13-47fc-9d96-240d0c117e41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:07.104648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eefdee-5c13-47fc-9d96-240d0c117e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:07.121626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-03eefdee-5c13-47fc-9d96-240d0c117e41 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:07.122826 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-03eefdee-5c13-47fc-9d96-240d0c117e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:07.131938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:07.167303 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc23f-7561-44e8-8eec-faa5e2a207f1 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:07.208603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:07.209369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:07.209589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:07.209862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:07.210551 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:07.211938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:07.218770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 453/910] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:22:07 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:22:07.219089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 458/911] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:07.237978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:07.306089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:07.347573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403b441-a205-48a7-9b35-511930cb6e1f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:07.385051 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5beda-7653-42ce-8d6b-b243f5e9381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:07.386025 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5beda-7653-42ce-8d6b-b243f5e9381d None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:07.386460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5beda-7653-42ce-8d6b-b243f5e9381d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:07.386910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5beda-7653-42ce-8d6b-b243f5e9381d None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:07.387795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5beda-7653-42ce-8d6b-b243f5e9381d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:07.389432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5beda-7653-42ce-8d6b-b243f5e9381d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:07.407324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 454/912] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:07 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:08.073647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da26c5-54c3-4091-ad57-2568674dd055 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.074198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da26c5-54c3-4091-ad57-2568674dd055 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.074198 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da26c5-54c3-4091-ad57-2568674dd055 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.075981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 459/913] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:08.082071 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea03c18-fbc5-460e-a331-e5b5c3cb7523 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.082376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea03c18-fbc5-460e-a331-e5b5c3cb7523 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.082629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea03c18-fbc5-460e-a331-e5b5c3cb7523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.100699 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-aea03c18-fbc5-460e-a331-e5b5c3cb7523 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:08.102259 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-aea03c18-fbc5-460e-a331-e5b5c3cb7523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:08.175979 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 455/914] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:08.198508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da26c5-54c3-4091-ad57-2568674dd055 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.254015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83495c5-a1fd-462d-91eb-9bc9c40a5c2f None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.254397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83495c5-a1fd-462d-91eb-9bc9c40a5c2f None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.254744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83495c5-a1fd-462d-91eb-9bc9c40a5c2f None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.257337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 456/915] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:22:08.261608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.261971 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.262309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.282168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:08.283911 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:08.307827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da26c5-54c3-4091-ad57-2568674dd055 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.338089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:08.343441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.344320 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.344596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.344915 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.358610 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:08.367589 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 457/916] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:08.370906 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:08.371834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:08.372717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:08.377186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 460/917] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:08.378643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.395063 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.409574 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:08.493291 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6e2ae-5850-4faa-acd5-596a857921f8 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.496974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756fbaa-03d5-42b5-9e39-afee9164a787 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.529873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.530800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.530991 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.531392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.532208 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:08.532577 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:08.534066 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:08.536103 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.537103 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.537579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.538019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.538843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:08.540395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:08.543283 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 458/918] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:08.550010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:08.557979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:08.559117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:22:08.559970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:08.569504 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 461/919] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:08.571680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.605690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:08.643551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:08.686258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79426c7-220a-4293-941f-ede4dba3552a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.686689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b831d2-51aa-4d1d-8cc7-d57ebba39d0a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.721726 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.722562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.722942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.723312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.724094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:08.724456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:08.725926 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:08.733382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 462/920] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:08.736953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.737727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.738218 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.738683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.739125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.753813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:08.756251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:08.779471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:08.780427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:08.781384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:08.781384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:08.781384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:08.781384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:08.787559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 459/921] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:08.788314 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.850053 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b917296-eaf1-4ece-92ca-25724ce500a9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.878809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:08.885278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.885953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.886233 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.886507 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.887213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:08.887520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:08.888939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:08.899775 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 463/922] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:08.901117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:08.922712 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:08.939738 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60da5d0-2163-46af-8fd9-6d0aecf410db None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:08.976101 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:08.976778 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:08.977063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:08.977357 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:08.978076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:08.978380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:08.979761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:08.997159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 460/923] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:08.998311 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.038468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7284d50-63a9-4f0f-8f6e-c70e971df169 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.073482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.074302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.074860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.074860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.075529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.076935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.109332 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 464/924] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:22:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:22:09.111123 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2877-1966-49a6-ab60-414acab58589 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.159575 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.160826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.161375 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.161858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.170864 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.172975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:09.191586 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:09.192668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:22:09.193682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:09.196706 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 461/925] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:22:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:09.203171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.261438 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:09.327899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b98ea-a6e2-4ff6-a4d8-130c03abd95f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.364140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.365092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.365543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.365959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.366997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:09.367460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.369216 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115f94a-0faf-47dc-90c1-c10d4f862fd4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.388317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 465/926] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:09.394217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.407116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:09.487524 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312749a-2cf4-482c-90c8-24650edc0129 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.487865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312749a-2cf4-482c-90c8-24650edc0129 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.488074 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312749a-2cf4-482c-90c8-24650edc0129 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.490080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 466/927] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:09.498631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d704db-7184-42f2-9f97-047613233c05 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.499225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d704db-7184-42f2-9f97-047613233c05 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.499700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d704db-7184-42f2-9f97-047613233c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.508574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f637156-f462-4160-b6b1-962a6bad7da8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.521425 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-96d704db-7184-42f2-9f97-047613233c05 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:09.523510 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-96d704db-7184-42f2-9f97-047613233c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:09.551199 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.552284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.552724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.553134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.564456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.566173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:09.587207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:09.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:09.589363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:09.589363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:09.589363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:09.589363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:09.591417 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 462/928] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:09.596739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.614872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 467/929] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:09.643390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d704db-7184-42f2-9f97-047613233c05 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.675566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:09.694228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:09.723836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d32a89-140c-43df-9dfb-d8db55416e43 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.756864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d704db-7184-42f2-9f97-047613233c05 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.760406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.761086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.761388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.761663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.762388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:09.762695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.764087 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6000c5-a068-4fe5-9a12-d7c4fcaec46b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.780802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 463/930] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:09.780802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa4b78-1f9a-480c-8c24-aab4a03759d6 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.780802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa4b78-1f9a-480c-8c24-aab4a03759d6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.780802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa4b78-1f9a-480c-8c24-aab4a03759d6 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.780802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 464/931] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:09.784639 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa4b78-1f9a-480c-8c24-aab4a03759d6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.799515 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.800618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.801072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.801408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.802059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.803420 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-aa30d5e9-8839-40bf-9ecf-bc7551260db7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:09.815608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:09.816587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:09.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 468/932] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:09.818839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.818839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.818839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.835200 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:09.836761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:09.899551 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa4b78-1f9a-480c-8c24-aab4a03759d6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:09.906513 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 469/933] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:09.907529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:09.940861 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:09.948480 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:09.957893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:09.958702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:09.959243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:09.959243 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:09.966382 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:09.968027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:09.982339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:09.983310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:09.984298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 465/934] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:09.984818 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.025904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15931bdb-f83a-417e-826b-1a924c2cb80c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.061072 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.061861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.062140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.062405 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.063080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:10.063395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.064769 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.073281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 470/935] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:10.073281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.096635 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ce63c-c5c5-4f10-a2b5-35add2d68d11 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.132566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.133409 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.133692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.133963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.134731 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.136217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:10.148898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:10.155746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:10.156669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 466/936] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:10.157550 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.177300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:10.183191 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30126a2a-bd54-4f34-bff9-11d0d866dcd0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.202668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:10.218742 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.219419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.219692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.220181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.220830 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:10.221044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.222510 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.234352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 471/937] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:10.235053 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.266033 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e994-a718-46e3-8f0d-b98b15000036 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.304712 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.305539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.305820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.306092 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.306813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:10.307121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.308520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.321792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 467/938] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:10.321792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.347364 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685c4d7-dd9b-4042-afb3-92dfb634c6a5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.380391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.381252 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.381601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.381962 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.382740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:10.383101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.384486 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.389500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 472/939] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:10.390324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.414783 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:10.434739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0591bef-d02b-49d3-be60-2e46427f7b54 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.443937 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:10.470726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.471627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.471906 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.472172 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/3124c05981b6416ca04be4e734434d0b/roles/f8e128c40c034a9587c922be6a3fc432` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.474499 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=3124c05981b6416ca04be4e734434d0b, role_id=f8e128c40c034a9587c922be6a3fc432)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.475962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.489904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 468/940] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:22:10 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/3124c05981b6416ca04be4e734434d0b/roles/f8e128c40c034a9587c922be6a3fc432 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:22:10.490908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.505634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e1709-70b5-475c-aca3-17af87582f68 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.562762 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.564104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.564545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.565182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.566955 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.569012 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.601789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 473/941] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:22:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:10.628765 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36529ac4-930d-48b5-9dc0-f559c09256db None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.680018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.680724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.681187 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.681682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.682489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:10.682864 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.684453 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.691252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 469/942] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:10.698482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.790408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:10.819672 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091ef5-297d-4722-829a-1185f9f4612b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:10.857597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:10.858285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:10.858603 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:10.858885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:10.859687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] RBAC: Adding query filter params (interface=public, service_id=e3a9625efbd349c69b5a125dd311101f, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:10.860000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:10.861415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f0ec1-feeb-4b47-804f-b6a8f5b054a7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:10.866919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 474/943] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:22:10 2022] GET /identity/v3/endpoints?service_id=e3a9625efbd349c69b5a125dd311101f&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:10.872629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:10.908229 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:10.985983 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e91dc-1d6d-4c1d-8e32-0107552407b4 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:11.021093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28319-5173-4696-a678-01befdac9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:11.022215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28319-5173-4696-a678-01befdac9e40 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:11.022559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28319-5173-4696-a678-01befdac9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:11.022909 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28319-5173-4696-a678-01befdac9e40 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:11.023666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad28319-5173-4696-a678-01befdac9e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:11.025107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad28319-5173-4696-a678-01befdac9e40 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:11.032026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 470/944] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:22:10 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:22:11.413760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e870c-2e46-4ebc-9458-bf4757ed9942 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:11.414298 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e870c-2e46-4ebc-9458-bf4757ed9942 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:11.414298 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e870c-2e46-4ebc-9458-bf4757ed9942 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:11.415858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 475/945] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:11.421606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:11.422129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:11.422601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:11.440264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:11.441503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:11.510596 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 471/946] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:11.534818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e870c-2e46-4ebc-9458-bf4757ed9942 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:11.638320 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e870c-2e46-4ebc-9458-bf4757ed9942 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:11.675298 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:11.675902 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:11.676181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:11.676464 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:11.677187 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:11.678648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:11.680078 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: WARNING keystone.server.flask.common [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:22:11.691959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:11.702579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.sql.core [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:22:11.702579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:22:11.702579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [parameters: {'id': '7fac93fe00a34537835cf61ca64eeae0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:22:11.702579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80298) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fac93fe00a34537835cf61ca64eeae0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fac93fe00a34537835cf61ca64eeae0. May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:22:11.710609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:22:11.711445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application yield resource May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise self.value May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application session.commit() May 19 14:22:11.712410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.flush() May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application compat.raise_( May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:22:11.713345 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:22:11.714268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:22:11.715314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [parameters: {'id': '7fac93fe00a34537835cf61ca64eeae0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:11.716238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:11.717068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fac93fe00a34537835cf61ca64eeae0. May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:11.721179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 476/947] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 14:22:11 2022] POST /identity/v3/projects => generated 262 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:11.721895 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:11.765896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:11.830405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935c2bf-9f58-46b5-ad91-9ed1066e6ab9 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:11.866184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:11.867013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:11.867294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:11.867569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:11.875926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:11.877466 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-f476ce55-59ec-4b6a-bcf8-c15217a9ab49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:11.892795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:11.893816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:11.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 472/948] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:11.900385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:12.007087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da4db-aafc-41f5-aec5-c08dfb3c03ba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:12.041166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:12.041871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:12.042184 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:12.042463 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:12.043221 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:12.043584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:12.045224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3547610-e838-4b68-ab46-c830c59f18d5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:12.069634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 477/949] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:12.868687 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd496c39-f527-46a5-bfb9-d86d527a4d85 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:12.869277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd496c39-f527-46a5-bfb9-d86d527a4d85 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:12.869277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd496c39-f527-46a5-bfb9-d86d527a4d85 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:12.870914 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 473/950] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:12.876333 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427196a6-79e5-418f-a8bf-8037959a03b1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:12.876847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427196a6-79e5-418f-a8bf-8037959a03b1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:12.876847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427196a6-79e5-418f-a8bf-8037959a03b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:12.894550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-427196a6-79e5-418f-a8bf-8037959a03b1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:12.896088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-427196a6-79e5-418f-a8bf-8037959a03b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:12.955960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:12.974472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 478/951] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:12.997923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd496c39-f527-46a5-bfb9-d86d527a4d85 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.009927 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88b1c-aceb-4368-9982-3e71ef8ff3ce None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.010514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88b1c-aceb-4368-9982-3e71ef8ff3ce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.010992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88b1c-aceb-4368-9982-3e71ef8ff3ce None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.012858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 479/952] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:13.017973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.018474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.018995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.039172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:13.040700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:13.096584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.120083 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd496c39-f527-46a5-bfb9-d86d527a4d85 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.126934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 480/953] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:13.151882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.156514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.157373 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.157664 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.157984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.169746 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:13.178511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:13.179482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:13.180377 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 474/954] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:22:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:13.185820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.265889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab82bb-1252-4385-9cd7-56647da96a64 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.287195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.299677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cd01c-1e08-4580-9d67-fcddde1898d3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.306888 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.307701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.308100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.308375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.309042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.310466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b9d97d24-ae3c-44ce-8a67-a1c0e2e4b37e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:13.322183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:13.323253 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:13.324143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 481/955] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:13.325010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d575b-e1ea-404c-8117-f5a0d380d8f4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.325010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d575b-e1ea-404c-8117-f5a0d380d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.325158 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d575b-e1ea-404c-8117-f5a0d380d8f4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.329182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 482/956] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:13.330140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d575b-e1ea-404c-8117-f5a0d380d8f4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.342471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.343192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.343474 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.343744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.344492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:13.344795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.346256 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c468e-d76b-48bf-a440-13509066d1dc None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.359471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 475/957] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:13.359471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.359471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.359471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.364769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.376827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:13.378348 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:13.445580 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d575b-e1ea-404c-8117-f5a0d380d8f4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.455863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 476/958] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:13.456619 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.486972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.487656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.487934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.488211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.488924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:13.489230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.490672 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.497644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 483/959] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:13.502492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.548932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.567975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fad4f-fb97-4490-b79f-56c5c2fbbe2e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.602573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.603381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.603662 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.603933 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.606113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.610432 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070ff5-16a3-4b74-801e-34ff574c46c4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.615010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.616473 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:13.633995 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:13.634921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:13.635850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:13.635850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:13.635850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:13.635850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:13.635850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 477/960] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:13.636852 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.651880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.652704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.652985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.653324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.654324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.655763 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:13.668374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:13.669733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:13.671110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 484/961] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:13.671937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.744061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03537748-bdea-406a-8a47-16f3efba0346 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.782412 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.783086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.783400 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.783647 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.784534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:13.784722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.786089 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.787734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136f993-e27c-4c8f-a267-13b29289b499 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.790823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 478/962] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:13.791629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.810329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.848244 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.849582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.850519 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:13.850994 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.851488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.852471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:13.852983 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.856215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.880621 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 485/963] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:13.881292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:13.916756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5dc2-9a42-467f-9e67-f4d7aef0b25a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:13.971407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:13.972585 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:13.973668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:13.974164 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:13.975336 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:13.975823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:13.978395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:13.984900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 479/964] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:13.991731 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.016227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057368e-bcc3-4968-9d09-810b3cac4075 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.060808 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.060808 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.061148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.061406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.070541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.072755 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:14.091885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:14.092991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:22:14.094041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:14.097359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 486/965] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:14.099788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.116197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca6959-b3c4-445d-8453-bfac5db62236 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.130186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.139467 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.167565 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.169090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.169848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.170340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.171474 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:14.172214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.174405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.181444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 480/966] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:14.189292 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.220496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd095f6-e18c-41cf-bdf2-6e6bd821d8e0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.265890 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.267197 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.267656 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.268262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.269728 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:14.270316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.271954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.296008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 487/967] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:22:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:14.302221 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.313773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb59c2-d81b-49af-862d-7eac69260554 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.349755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a1bce-42c8-48c0-ab13-956c99e1bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.350618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a1bce-42c8-48c0-ab13-956c99e1bbbd None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.350993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a1bce-42c8-48c0-ab13-956c99e1bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.351363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a1bce-42c8-48c0-ab13-956c99e1bbbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.352270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a1bce-42c8-48c0-ab13-956c99e1bbbd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.353811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a1bce-42c8-48c0-ab13-956c99e1bbbd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.386776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 481/968] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:22:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:22:14.393908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5464-c32b-4592-9ba8-a81dbc74775d None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.394352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5464-c32b-4592-9ba8-a81dbc74775d None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.395241 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5464-c32b-4592-9ba8-a81dbc74775d None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.396678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 482/969] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:14.402200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.402633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.403059 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.408849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.422539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:14.423701 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:14.431452 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca54d0-149d-4bbf-9174-d982062b3505 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.455217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.480541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.481300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.481624 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.481894 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.523773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 483/970] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:14.549680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.583910 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.585476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.670272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25247d48-2b8c-4b1c-b23e-d08054f6a7a6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.686760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.712706 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 488/971] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:14 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:14.718141 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.719091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.719786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.720070 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.720344 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.721279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.723638 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.724992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: WARNING keystone.server.flask.common [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:22:14.741422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.sql.core [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:22:14.741422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:22:14.741422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [parameters: {'user_id': '373a81587f2f48cebeb08916fb03cdaf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:22:14.741422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80299) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application conn.query(q) May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application result.read() May 19 14:22:14.761921 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:22:14.762874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application next(self.gen) May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application yield resource May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application next(self.gen) May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise self.value May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application session.commit() May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:22:14.763844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self.session.flush() May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application compat.raise_( May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:22:14.764947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:22:14.765908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application conn.query(q) May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application result.read() May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:22:14.766768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [parameters: {'user_id': '373a81587f2f48cebeb08916fb03cdaf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:14.767622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:22:14.768427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:22:14.770819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:22:14.770819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:22:14.770819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:22:14.770819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:22:14.770819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:22:14.770819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:14.773537 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 484/972] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:22:14 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:14.779779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.812799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.834246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543ae22-fde0-4219-b49b-362383ea14e3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.870112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.870849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.871179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.871458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.882172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.883638 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.897465 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e75bcc-403f-4e5a-83fb-322005e08c97 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:14.902202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:14.903107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:14.904416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:14.904416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:14.904416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:14.904416 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:14.908598 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 489/973] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:14.909543 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:14.940543 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:14.942258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:14.942763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:14.943233 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:14.948806 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:14.953779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:14.955681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:14.974079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:14.975112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:22:14.976104 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:14.976659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 485/974] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:22:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:14.981478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:15.024972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a1f5-95b4-44a8-b75e-7ff8e6f297a0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:15.063096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:15.064272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:15.064553 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:15.064830 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:15.066046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:15.066369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:15.068183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:15.074893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:15.089393 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 490/975] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:15.091247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:15.097890 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1256ae-27ed-4fee-b79d-1317353c20c9 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:15.137873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:15.138555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:15.138831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:15.139102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:15.139945 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:15.140249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:15.141777 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:15.189269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 486/976] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:22:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:15.209650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:15.217533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19116e03-97f7-4095-86f7-2096a45b392f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:15.253771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:15.254591 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:15.254870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:15.255142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:15.262968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:15.264525 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:15.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:15.281930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:15.282874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 491/977] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:15.290444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:15.401301 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db829a-8ff3-493e-b4a6-3bf66296d65b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:15.438167 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:15.439185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:15.439671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:15.440145 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:15.441149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:15.441496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:15.442945 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720329e-1743-417f-8cbe-165260d32d25 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:15.455741 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 487/978] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:15.461920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:15.567493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8bc1f-370f-4812-9b84-ea6c93361b76 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:15.603676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:15.604346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:15.604618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:15.604892 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:15.605762 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] RBAC: Adding query filter params (role.id=f8e128c40c034a9587c922be6a3fc432, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=3124c05981b6416ca04be4e734434d0b) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:15.606083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:15.607522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae24b7-7d58-4bae-9c25-3efa79ee90e7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:15.613905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 492/979] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:15 2022] GET /identity/v3/role_assignments?user.id=3124c05981b6416ca04be4e734434d0b&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=f8e128c40c034a9587c922be6a3fc432 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:16.088640 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165711-31fa-40ba-a3f2-ac2fd1f539ac None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:16.089189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165711-31fa-40ba-a3f2-ac2fd1f539ac None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:16.089189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165711-31fa-40ba-a3f2-ac2fd1f539ac None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:16.090575 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 488/980] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:16.096286 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cd399-30e4-42c4-8bbd-10989294d22a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:16.096717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cd399-30e4-42c4-8bbd-10989294d22a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:16.096940 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cd399-30e4-42c4-8bbd-10989294d22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:16.115262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a77cd399-30e4-42c4-8bbd-10989294d22a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:16.116468 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a77cd399-30e4-42c4-8bbd-10989294d22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:16.131252 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:16.191919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 493/981] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:16.214207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a165711-31fa-40ba-a3f2-ac2fd1f539ac None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:16.218282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:16.320581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a165711-31fa-40ba-a3f2-ac2fd1f539ac None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:16.356429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a793ac-fa85-4194-995b-1fff14bc561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:16.357097 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a793ac-fa85-4194-995b-1fff14bc561f None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:16.357434 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a793ac-fa85-4194-995b-1fff14bc561f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:16.357707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a793ac-fa85-4194-995b-1fff14bc561f None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:16.358513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a793ac-fa85-4194-995b-1fff14bc561f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:16.359901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a793ac-fa85-4194-995b-1fff14bc561f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:16.378190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 489/982] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:16 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:16.689749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a66b9-4d3b-42b2-896e-1a22d11acc79 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:16.690034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a66b9-4d3b-42b2-896e-1a22d11acc79 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:16.690315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a66b9-4d3b-42b2-896e-1a22d11acc79 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:16.691704 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 494/983] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:16.697773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:16.698066 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:16.698342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:16.715952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:16.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:16.778770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:16.796905 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 490/984] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:16.819175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a66b9-4d3b-42b2-896e-1a22d11acc79 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:16.926504 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a66b9-4d3b-42b2-896e-1a22d11acc79 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:16.963352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:16.964439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:16.964892 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:16.965385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:16.966211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:16.967876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:16.980273 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:16.981323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:16.982192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:16.985400 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 495/985] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:16.988889 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:17.090755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75fba4-dd59-4efe-8699-357bca5c7fbe None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:17.125251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:17.125935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:17.126207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:17.126472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:17.127167 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:17.127469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:17.128858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:17.134103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 491/986] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:17.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:17.156781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:17.246044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6e8c-2d48-47eb-b8e5-b0394b3c7182 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:17.281136 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:17.281834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:17.282106 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:17.282427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:17.283140 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:17.283443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:17.285031 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:17.294989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 496/987] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:17.296025 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:17.352736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:17.398340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076224f-bb25-45b0-b05f-80f3bdb9ae39 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:17.434566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:17.435232 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:17.435470 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:17.435775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:17.436688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] RBAC: Adding query filter params (interface=admin, service_id=e3a9625efbd349c69b5a125dd311101f, region_id=RegionOne) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:17.437048 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:17.438471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a4655-f373-402c-8c6c-7222a12f5a8b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:17.443940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 492/988] 10.222.0.73 () {56 vars in 1249 bytes} [Thu May 19 14:22:17 2022] GET /identity/v3/endpoints?service_id=e3a9625efbd349c69b5a125dd311101f&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:17.449464 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:17.537896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:17.563471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd898b-247a-434c-8015-73a80057bcc9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:17.599145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4700d7-2c46-4336-ae07-6e988a90e223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:17.599831 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4700d7-2c46-4336-ae07-6e988a90e223 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:17.600095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4700d7-2c46-4336-ae07-6e988a90e223 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:17.600361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4700d7-2c46-4336-ae07-6e988a90e223 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:17.601046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4700d7-2c46-4336-ae07-6e988a90e223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:17.602458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4700d7-2c46-4336-ae07-6e988a90e223 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:17.609728 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 497/989] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:22:17 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:22:17.858233 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc41026-ee60-4d3f-a7bf-0134be567df6 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:17.858681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc41026-ee60-4d3f-a7bf-0134be567df6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:17.858906 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc41026-ee60-4d3f-a7bf-0134be567df6 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:17.860319 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 493/990] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:17.865822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:17.866081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:17.866384 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:17.883472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:17.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:17.964205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 498/991] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:17.986579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc41026-ee60-4d3f-a7bf-0134be567df6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:18.094779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc41026-ee60-4d3f-a7bf-0134be567df6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:18.112743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:18.131848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:18.132668 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:18.132954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:18.133212 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:18.133910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:18.135294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:18.147133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:18.148180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:18.154636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 494/992] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:22:17 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:18.157495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:18.229573 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:18.262928 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645f7c5-981a-4010-96db-491b2292c696 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:18.298244 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:18.299216 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:18.299662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:18.300078 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:18.300956 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:18.301463 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:18.302999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d837e5-9fd0-4095-8203-dee1581705c2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:18.308366 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 499/993] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:18.314401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:18.419426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9bd62-0272-44d7-a302-a48134cd74c3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:18.455050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:18.455743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:18.456019 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:18.456290 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:18.457022 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] RBAC: Adding query filter params (type=image) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:18.457362 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:18.458811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44817532-195f-49ef-8a2c-1b87b50a8061 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:18.464626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 495/994] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:18.651410 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f3413-5a76-4723-9d9a-e99d543aa614 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:18.651831 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f3413-5a76-4723-9d9a-e99d543aa614 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:18.652244 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f3413-5a76-4723-9d9a-e99d543aa614 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:18.653859 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 500/995] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:18.660068 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:18.660352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:18.660726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:18.678445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:18.679560 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:18.721554 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:18.747978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 496/996] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:18.770692 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f3413-5a76-4723-9d9a-e99d543aa614 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:18.877176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f3413-5a76-4723-9d9a-e99d543aa614 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:18.923514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:18.924654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:18.924933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:18.925204 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:18.925902 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:18.927290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:18.929626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:18.945933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:18.946963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:18.952919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 501/997] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:18.955309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:19.061046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f535-69c4-4e6c-8f9b-acfad059ea57 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:19.095117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:19.095768 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:19.096062 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:19.096329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:19.097036 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:19.097359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:19.098761 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:19.103843 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 497/998] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:19.109631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:19.213781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d7292-bc5e-4760-9a1d-d7a265d9ffb1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:19.247314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:19.247991 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:19.248270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:19.248545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:19.249292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:19.249618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:19.251000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:19.256304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 502/999] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:19.262076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:19.306973 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:19.367235 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd6a00-b9c8-4ff1-858d-a9ae4246e22d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:19.402451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:19.403116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:19.403392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:19.403659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:19.404811 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] RBAC: Adding query filter params (interface=public, service_id=d5df0aefe2fa41a08474d9b48724b61c, region_id=RegionOne) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:19.405119 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:19.406584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a23ce-cda1-41cc-8ea8-07fb84bf102e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:19.411919 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 498/1000] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:22:19 2022] GET /identity/v3/endpoints?service_id=d5df0aefe2fa41a08474d9b48724b61c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:19.417529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:19.478589 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:19.519872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10bec2-8fd4-4567-a523-20c8b2488992 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:19.554880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28dff-4f5c-4ad8-ba9b-836f15fc460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:19.555563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28dff-4f5c-4ad8-ba9b-836f15fc460b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:19.555809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28dff-4f5c-4ad8-ba9b-836f15fc460b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:19.556089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28dff-4f5c-4ad8-ba9b-836f15fc460b None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:19.556768 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28dff-4f5c-4ad8-ba9b-836f15fc460b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:19.558165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28dff-4f5c-4ad8-ba9b-836f15fc460b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:19.564788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 503/1001] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:22:19 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:22:19.867940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698cdeb-99e9-4656-b1cc-aac2ab216a8e None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:19.868311 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698cdeb-99e9-4656-b1cc-aac2ab216a8e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:19.868492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698cdeb-99e9-4656-b1cc-aac2ab216a8e None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:19.869967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 499/1002] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:19.875406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:19.875689 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:19.875963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:19.893279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:19.894481 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:19.969136 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 504/1003] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:19.992015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698cdeb-99e9-4656-b1cc-aac2ab216a8e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:20.099129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698cdeb-99e9-4656-b1cc-aac2ab216a8e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:20.101540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:20.135870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:20.136596 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.136865 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.137128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.137796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:20.139183 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:20.151512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:20.152576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:20.159171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 500/1004] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:22:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:20.160125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:20.216157 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:20.261732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa645223-8d57-4672-ba20-5fde21b2e403 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:20.302184 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:20.302854 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.303108 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.303392 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.304094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:20.304391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:20.305788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:20.315710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 505/1005] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:20.316955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:20.420126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940dede-0f73-4336-9973-09ea5b66ac66 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:20.453088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:20.453812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.454092 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.454359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.455060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:20.455359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:20.456726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19522239-84b6-4040-89ac-8b79046c5e13 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:20.466111 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 501/1006] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:22:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:20.467399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:20.574497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41508f12-1937-4af0-b2ec-55672f93fc70 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:20.594954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:20.612231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f77b1-4abb-4d6e-a22b-cf3fcfdbfae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:20.612897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f77b1-4abb-4d6e-a22b-cf3fcfdbfae0 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.613179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f77b1-4abb-4d6e-a22b-cf3fcfdbfae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.613474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f77b1-4abb-4d6e-a22b-cf3fcfdbfae0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.614217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f77b1-4abb-4d6e-a22b-cf3fcfdbfae0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:20.615578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f77b1-4abb-4d6e-a22b-cf3fcfdbfae0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:20.631408 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 506/1007] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:22:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:22:20.681142 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947180-c058-4e42-982a-4a64c36dce49 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.681500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947180-c058-4e42-982a-4a64c36dce49 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.681670 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947180-c058-4e42-982a-4a64c36dce49 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.683404 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 502/1008] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:20.689722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac14239-f62d-411d-a853-167916060387 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.690080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac14239-f62d-411d-a853-167916060387 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.690290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac14239-f62d-411d-a853-167916060387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.710618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bac14239-f62d-411d-a853-167916060387 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:20.710915 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bac14239-f62d-411d-a853-167916060387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:20.792952 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 507/1009] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:20.815853 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947180-c058-4e42-982a-4a64c36dce49 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:20.849500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:20.929329 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947180-c058-4e42-982a-4a64c36dce49 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:20.965902 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5ab6d-91b6-4608-9358-81e3ffa17e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:20.966576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5ab6d-91b6-4608-9358-81e3ffa17e8d None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:20.966856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5ab6d-91b6-4608-9358-81e3ffa17e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:20.967130 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5ab6d-91b6-4608-9358-81e3ffa17e8d None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:20.967898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5ab6d-91b6-4608-9358-81e3ffa17e8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:20.969320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5ab6d-91b6-4608-9358-81e3ffa17e8d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:20.983031 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 503/1010] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:20 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:21.802955 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1d49e-f0b6-4cad-becc-025f8591c4ec None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:21.803520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1d49e-f0b6-4cad-becc-025f8591c4ec None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:21.803520 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1d49e-f0b6-4cad-becc-025f8591c4ec None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:21.804864 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 508/1011] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:21.810736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:21.811176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:21.811256 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:21.828664 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:21.829899 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:21.906862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 504/1012] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:21 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:21.929680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1d49e-f0b6-4cad-becc-025f8591c4ec None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:22.033159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1d49e-f0b6-4cad-becc-025f8591c4ec None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:22.040361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:22.069430 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:22.070165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:22.070441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:22.070707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:22.071369 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:22.072743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:22.084343 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:22.085661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:22:22.086531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:22.089635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 509/1013] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:22.091681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:22.111869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:22.194608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed36041-36f7-4966-8b23-a34761137219 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:22.228680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:22.229402 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:22.229694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:22.229967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:22.230677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:22.230981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:22.232367 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:22.237833 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 505/1014] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:22.243385 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:22.343885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6371eb-6332-4346-bac8-64fb06e3d0c9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:22.377170 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:22.377882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:22.378162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:22.378435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:22.379146 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:22.379446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:22.380821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881d448-b813-4e35-ba38-284410be8f28 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:22.385982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 510/1015] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:22:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:22.391559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:22.474179 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:22.493599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cd9cf-8fcb-406d-b0b3-af7d6b8c590f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:22.528210 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410bfd7-3a36-40cb-9015-78f4e34f8be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:22.528872 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410bfd7-3a36-40cb-9015-78f4e34f8be4 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:22.529621 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410bfd7-3a36-40cb-9015-78f4e34f8be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:22.529881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410bfd7-3a36-40cb-9015-78f4e34f8be4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:22.530854 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410bfd7-3a36-40cb-9015-78f4e34f8be4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:22.532242 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410bfd7-3a36-40cb-9015-78f4e34f8be4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:22.552998 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 506/1016] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:22:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:22.908730 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3fd50-91ac-4f41-a42d-81df4f472e55 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:22.909326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3fd50-91ac-4f41-a42d-81df4f472e55 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:22.909326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3fd50-91ac-4f41-a42d-81df4f472e55 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:22.910874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 511/1017] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:22.917321 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:22.917321 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:22.917321 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:22.937491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:22.938851 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:23.015929 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 507/1018] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:23.040007 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3fd50-91ac-4f41-a42d-81df4f472e55 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:23.079995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:23.152830 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3fd50-91ac-4f41-a42d-81df4f472e55 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:23.188748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1857d7-9b07-423e-bd7e-432651efb75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:23.189566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1857d7-9b07-423e-bd7e-432651efb75f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.189847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1857d7-9b07-423e-bd7e-432651efb75f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.190117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1857d7-9b07-423e-bd7e-432651efb75f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.191291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1857d7-9b07-423e-bd7e-432651efb75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:23.192735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1857d7-9b07-423e-bd7e-432651efb75f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:23.198948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 512/1019] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:23.200252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:23.203574 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8211-5a68-446a-9403-44866f942c0e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.203865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8211-5a68-446a-9403-44866f942c0e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.204152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8211-5a68-446a-9403-44866f942c0e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.205652 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 513/1020] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:22:23.210303 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.210587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.210862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.228668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:23.229877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:23.264335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:23.297502 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 514/1021] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:23.308597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c3dd-a47d-4bdd-a6a0-5b218c1553dd None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:23.319572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:23.345413 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:23.346103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.346384 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.346652 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.347372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:23.348825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:23.351209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:23.375339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 508/1022] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:23 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:23.428144 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b2af6-a540-4068-9c9d-985f44fa3114 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:23.466986 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:23.467751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.468040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.468295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.468989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:23.470431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:23.486407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:23.487576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:23.488474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:23.494796 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 515/1023] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:22:23 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:23.495009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:23.608930 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4057e-a2b5-4075-81cb-dcdd0a5d97ae None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:23.626704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:23.647584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:23.648264 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.648548 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.648820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.649930 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] RBAC: Adding query filter params (name=image) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:23.650401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:23.652374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:23.658013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 509/1024] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:23.663919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:23.772041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:23.779870 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b095fa-843f-4be7-9b8f-858197708706 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:23.815919 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:23.816593 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.816923 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.817209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.817954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] RBAC: Adding query filter params (type=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:23.818252 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:23.819619 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:23.825056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 516/1025] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:23.831978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:23.947916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b8b15-5065-4122-a3ae-94386d542d36 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:23.985845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:23.986572 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:23.986853 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:23.987122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:23.987918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] RBAC: Adding query filter params (interface=public, service_id=eb8a73a33f024ad6b92f58465e357e43, region_id=RegionOne) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:23.988305 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:23.989717 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafee35-20db-4432-99e0-dc6741b1526e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:24.000030 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 510/1026] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:22:23 2022] GET /identity/v3/endpoints?service_id=eb8a73a33f024ad6b92f58465e357e43&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:24.001049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:24.105635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091de07-5eb7-43f4-a9aa-ff6296219e83 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:24.140848 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965d9cf-ec20-4958-8748-a5cdc3be2727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:24.141547 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965d9cf-ec20-4958-8748-a5cdc3be2727 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:24.141803 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965d9cf-ec20-4958-8748-a5cdc3be2727 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:24.142096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965d9cf-ec20-4958-8748-a5cdc3be2727 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:24.142787 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965d9cf-ec20-4958-8748-a5cdc3be2727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:24.144172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965d9cf-ec20-4958-8748-a5cdc3be2727 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:24.151066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 517/1027] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:22:23 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:22:24.906012 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec4c2-7644-4a3c-99eb-7e060589ff5f None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:24.906274 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec4c2-7644-4a3c-99eb-7e060589ff5f None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:24.906559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec4c2-7644-4a3c-99eb-7e060589ff5f None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:24.908166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 511/1028] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:24.913503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:24.913789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:24.914057 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:24.931490 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:24.933287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:24.955071 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:25.003114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 518/1029] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:25.025951 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec4c2-7644-4a3c-99eb-7e060589ff5f None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.076725 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:25.135888 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec4c2-7644-4a3c-99eb-7e060589ff5f None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:25.170212 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:25.171130 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.171403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.171671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.181890 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:25.190176 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:25.191166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:25.192029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 512/1030] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:25.197461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.301136 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e70bcb-1670-47c5-a96d-cbaad278f5ad None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:25.336354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:25.337031 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.337338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.337609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.338355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:25.338658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:25.340037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:25.348640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 519/1031] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:25.354184 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.456799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89eac4-ed8c-45ef-98ae-99f1ea41b06a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:25.459564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:25.495464 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:25.496271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.496549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.496813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.507182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:25.508698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a99ef4ab-6ef7-4b8e-a639-d77cb3d3482c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:25.526240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:25.527158 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:25.528088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:25.528088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:25.528088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:25.528088 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:25.529563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 513/1032] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:25.533680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.552954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:25.635739 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587936b-5429-43ee-b643-e5aabbc1e98d None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.636048 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587936b-5429-43ee-b643-e5aabbc1e98d None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.636321 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587936b-5429-43ee-b643-e5aabbc1e98d None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.637882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 514/1033] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:25.640305 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fc55f-ba68-4293-978d-e95005455a5a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:25.643078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.643427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.643764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.665493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:25.666806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:25.680562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:25.681229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.681535 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.681805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.682527 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:25.682969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:25.684208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:25.701458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 520/1034] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:25.702814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.749684 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 515/1035] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:25.769599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.784130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:25.808497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad48a-7cfa-4880-8a5b-ba3f8bee0b22 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:25.849262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:25.850099 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.850375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.850642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.855974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:25.862380 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:25.865121 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:25.871467 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cf75c-56e2-47a2-82b5-94366bdbbc55 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:25.884716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:25.885897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:22:25.887032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:25.888987 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 521/1036] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:25.894798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:25.914091 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:25.915265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:25.915729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:25.916182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:25.919239 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:25.921943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:25.924135 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 516/1037] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:25.929827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.008062 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f18b19-9789-4540-b987-9566eaf870bb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.038643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8ca65-c6b8-44a7-b0fb-027ca2a6f671 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.045803 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.046895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.047179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.047450 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.048289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:26.048948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.050733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.054578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.073347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 522/1038] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:22:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:26.079224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.086575 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75de7c-97a4-4da5-bb25-3ec35781c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.087258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75de7c-97a4-4da5-bb25-3ec35781c47d None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.087539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75de7c-97a4-4da5-bb25-3ec35781c47d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.087807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75de7c-97a4-4da5-bb25-3ec35781c47d None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.088521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75de7c-97a4-4da5-bb25-3ec35781c47d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.091129 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75de7c-97a4-4da5-bb25-3ec35781c47d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.125292 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 517/1039] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:22:25 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:22:26.197970 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda53838-fdac-456b-a565-9a32aed41cee None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.239231 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.239913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.240193 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.240467 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.309472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.318304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.319769 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.389307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3b588-74ff-49e2-8680-619a98e45aa0 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.389651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3b588-74ff-49e2-8680-619a98e45aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.389870 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3b588-74ff-49e2-8680-619a98e45aa0 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.391513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 518/1040] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:26.396527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.396808 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.397082 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.415848 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:26.417052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:26.420358 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.428206 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 523/1041] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:22:26.433988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.493623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 519/1042] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:26.511712 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.531142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.539092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac387e-f23a-4f13-a8d6-e438e9468428 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.575593 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.576307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.576584 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.576885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.587671 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.589340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:26.609773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:26.610770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:26.616338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:26.616338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:26.616338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:26.616338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:26.616570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04856612-7d02-4848-bcb5-7cb71903bf99 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.616646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 524/1043] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:26.616646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.653573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.654757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.655127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.655391 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] PATH_INFO: `/v3/services/image` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.656065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.658877 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:26.670010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:26.671156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:26.674240 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 520/1044] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:26.677274 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.681048 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.731472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b520-37c2-42dd-84af-3056fd10c055 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.765884 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.766675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.766958 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.767205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.767920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:26.768218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.769609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.786397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 525/1045] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:26.787396 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.793637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86cbef-fcd8-4995-b43c-2f5fac4966da None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.796473 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.831405 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.831990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.832277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.832566 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.833279 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:26.833579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.834964 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.844550 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 521/1046] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:26.846060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.896161 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a03bcf-b425-4c91-a33d-e89dd6e77600 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.911682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:26.944013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.944826 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:26.945107 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:26.945423 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:26.952509 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:26.953992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:26.954997 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee740d99-cc1f-4388-b675-340be0fd4ec2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:26.969050 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:26.970296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:26.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 526/1047] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:26.975922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:26.999150 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:26.999826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:27.000094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:27.000373 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:27.001086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] RBAC: Adding query filter params (type=image) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:27.001506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:27.002826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:27.008581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 522/1048] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:27.014501 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:27.046816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:27.086489 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d6fb3-0015-4afa-8c54-7175f5917caf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:27.121977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e836b28-6f5f-4647-8ea9-357d0ef70c8e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:27.122914 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:27.123579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:27.123911 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:27.124183 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:27.124969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:27.125293 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:27.126723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:27.138717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 527/1049] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:27.144244 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:27.159200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:27.164383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc9c31-e42b-46e0-9815-b3e5a8fa43f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:27.164881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9c31-e42b-46e0-9815-b3e5a8fa43f4 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:27.165520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9c31-e42b-46e0-9815-b3e5a8fa43f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:27.166144 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9c31-e42b-46e0-9815-b3e5a8fa43f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:27.166994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9c31-e42b-46e0-9815-b3e5a8fa43f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:27.168363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9c31-e42b-46e0-9815-b3e5a8fa43f4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:27.191594 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 523/1050] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:22:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:27.263436 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77639ae-6dbd-4a88-ba4c-c1365e2486c1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:27.291758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:27.310111 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:27.310801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:27.311077 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:27.311420 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:27.312295 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] RBAC: Adding query filter params (role.id=f8e128c40c034a9587c922be6a3fc432, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=b4d24bc13b5c469cade02763acf072bd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:27.312608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:27.314154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01f1b5-90e2-44b0-a570-bf77927ce5dc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:27.321076 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 528/1051] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:27 2022] GET /identity/v3/role_assignments?user.id=b4d24bc13b5c469cade02763acf072bd&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=f8e128c40c034a9587c922be6a3fc432 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:28.406236 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8537ed2-b863-4563-9b83-df3686cef971 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:28.406793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8537ed2-b863-4563-9b83-df3686cef971 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:28.407050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8537ed2-b863-4563-9b83-df3686cef971 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:28.408440 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 524/1052] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:28.414561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:28.415101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:28.415101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:28.432539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:28.433779 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:28.503243 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 529/1053] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:28.526495 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8537ed2-b863-4563-9b83-df3686cef971 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:28.631271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8537ed2-b863-4563-9b83-df3686cef971 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:28.665288 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:28.666028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:28.666306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:28.666576 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:28.678219 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:28.685787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:28.686887 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:28.687757 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:28.689317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 525/1054] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:28.695572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:28.796258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5d3a7-bf7b-4dc4-8e0e-e2bfbc3ef78d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:28.813255 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:28.833504 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:28.834187 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:28.834465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:28.834738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:28.835493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:28.835802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:28.837187 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:28.845972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 530/1055] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:28.851637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:28.902931 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:28.961869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d088-e934-43d9-9a54-cf4dc790e39f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:28.996341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:28.997315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:28.997587 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:28.997901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] PATH_INFO: `/v3/users/abf63cf02f894c55ae2fa99021ee2b12` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.011939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] RBAC: Authorizing `identity:get_user(user_id=abf63cf02f894c55ae2fa99021ee2b12)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:29.013454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:29.015410 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 526/1056] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:28 2022] GET /identity/v3/users/abf63cf02f894c55ae2fa99021ee2b12 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:29.021171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:29.127685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb885de-4eeb-4dd9-a0e4-3ae6a3745d1f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:29.162670 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde7f50-2097-487f-898c-4e39df11d373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:29.163329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde7f50-2097-487f-898c-4e39df11d373 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.163607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde7f50-2097-487f-898c-4e39df11d373 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.163875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde7f50-2097-487f-898c-4e39df11d373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.210980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:29.236227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde7f50-2097-487f-898c-4e39df11d373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:29.237767 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde7f50-2097-487f-898c-4e39df11d373 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:29.348668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 531/1057] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:29.354485 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:29.418091 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861972a-dd35-4b7b-a007-2001eb0d38ae None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.418650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861972a-dd35-4b7b-a007-2001eb0d38ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.418650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861972a-dd35-4b7b-a007-2001eb0d38ae None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.419969 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 532/1058] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:22:29.424935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5cd97-82aa-4050-b535-863313931a1e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.425212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5cd97-82aa-4050-b535-863313931a1e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.425526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5cd97-82aa-4050-b535-863313931a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.448675 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-98f5cd97-82aa-4050-b535-863313931a1e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:29.449893 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-98f5cd97-82aa-4050-b535-863313931a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:29.466420 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa74b-ba3c-4147-ae57-af58768abacd None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:29.474180 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:29.504056 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e225c9-d337-4942-9a14-c7709d7481de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:29.504923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e225c9-d337-4942-9a14-c7709d7481de None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.505200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e225c9-d337-4942-9a14-c7709d7481de None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.505503 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e225c9-d337-4942-9a14-c7709d7481de None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.506677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e225c9-d337-4942-9a14-c7709d7481de None admin] RBAC: Authorizing `identity:get_project(project_id=35baa6c59d80476f84b1a823765168d0)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:29.508133 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e225c9-d337-4942-9a14-c7709d7481de None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:29.509952 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 527/1059] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/projects/35baa6c59d80476f84b1a823765168d0 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:29.511248 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918251a0-a7b3-4ebe-97a5-d55a58c2ad51 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.511556 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918251a0-a7b3-4ebe-97a5-d55a58c2ad51 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.511805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918251a0-a7b3-4ebe-97a5-d55a58c2ad51 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.513109 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 528/1060] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:29.515796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918251a0-a7b3-4ebe-97a5-d55a58c2ad51 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:29.523710 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 533/1061] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:22:29.530441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.530441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.530441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.542046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:29.543223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:29.545171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:29.609195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 534/1062] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:29.610056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:29.615299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918251a0-a7b3-4ebe-97a5-d55a58c2ad51 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:29.650236 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:29.651035 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.651388 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.651692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.652737 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=35baa6c59d80476f84b1a823765168d0, user.id=abf63cf02f894c55ae2fa99021ee2b12) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:29.653083 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:29.655248 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:29.673267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 529/1063] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/role_assignments?user.id=abf63cf02f894c55ae2fa99021ee2b12&scope.project.id=35baa6c59d80476f84b1a823765168d0&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:29.673267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:29.712513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:29.733466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad312532-b95d-485c-8038-515c6f8d17f5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:29.775853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:29.776526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.776846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.777111 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.787577 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:29.788496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d0a7-3205-4fbb-ba2c-cc0226f1d215 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:29.835390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:29.836235 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:29.836516 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:29.836786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:29.847807 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:29.856005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:29.856982 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:29.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 530/1064] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:29.861443 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:29.863115 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:29.864021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:29.964221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:29.968451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cf22d-2f22-4a6a-8303-e2cced5573d2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:29.974516 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 535/1065] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:29.980586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.006637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.007223 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.007505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.007772 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.008518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:30.008820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.010237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.018856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 531/1066] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:30.021446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:30.024407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.089968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932519ae-ecf5-45cb-87c4-662c4d724934 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.127588 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894c6b1-171b-4ab9-8da1-ca11c34591a2 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.130697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.131525 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.131804 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.132073 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.143942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.145448 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:30.166588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:30.167721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:30.168654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:30.168654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:30.168654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:30.168654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:30.168654 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 536/1067] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:30.170250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.171369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.171790 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.172189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.173699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.186059 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.188203 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:30.209948 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:30.210910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:30.211855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:30.211855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:30.211855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:30.211855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:30.216026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 532/1068] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:30.216717 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.232666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:30.267726 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:30.278498 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc77c0d-8787-49f7-b6f0-5eed24c91c57 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.315683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.316458 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.316738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.317010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.317772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:30.318082 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.319460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.328872 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedffb0-dc5f-44b3-afaa-a87d68b5524b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.331816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 537/1069] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:30.366535 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.367233 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.367497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.367767 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.368499 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:30.368803 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.370272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.383913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 533/1070] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:30.389658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.504361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecea3f-a4a8-4116-823f-d0d95eeb2b53 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.543145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.543882 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.544162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.544431 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.553032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.554549 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:30.568733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:30.569865 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:30.570938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 538/1071] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:30.575796 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.653515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:30.684528 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678547a-7fbd-4828-8e00-f2c70f877def None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.719309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.720024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.720294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.720562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.721428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:30.721732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.723201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.741205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 534/1072] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:30.746964 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:30.764167 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:30.854749 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0a110-23af-47e2-85f4-80389cd1fd10 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:30.889596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:30.890389 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:30.890669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:30.890941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:30.901822 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:30.903282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:30.920980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:30.922066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:30.923124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:30.923124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:30.923124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:30.923124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:30.923124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 539/1073] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:30.928102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:31.025350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd42d-da72-458d-91b3-080b658988b3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:31.054284 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:31.060713 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:31.061462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:31.061740 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:31.062011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:31.062728 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:31.063032 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:31.064417 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:31.075368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 535/1074] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:31.081178 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:31.143561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:31.183559 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608300c8-6f95-4dac-8f1a-286ac825bb38 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:31.217833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:31.218632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:31.218905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:31.219174 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:31.225964 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:31.227395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:31.241701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:31.242697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:31.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 540/1075] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:31.248881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:31.351427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb0336-e97c-4a8a-8950-860119a7a8ed None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:31.387246 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:31.387915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:31.388196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:31.388472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:31.389298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:31.389628 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:31.391084 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39acf3-ecd9-4a66-b0d6-e21fe5c29f17 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:31.403464 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 536/1076] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:31.409832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:31.515623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98fb9b-b41b-48e7-acc4-da13352a2725 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:31.539947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:31.553126 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8d357-c8ab-4739-91ad-59bb1c39b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:31.554064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d357-c8ab-4739-91ad-59bb1c39b2fa None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:31.554344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d357-c8ab-4739-91ad-59bb1c39b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:31.554617 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d357-c8ab-4739-91ad-59bb1c39b2fa None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/b4d24bc13b5c469cade02763acf072bd/roles/f8e128c40c034a9587c922be6a3fc432` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:31.556974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8d357-c8ab-4739-91ad-59bb1c39b2fa None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=b4d24bc13b5c469cade02763acf072bd, role_id=f8e128c40c034a9587c922be6a3fc432)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:31.558446 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8d357-c8ab-4739-91ad-59bb1c39b2fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:31.572368 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 541/1077] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:22:31 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/b4d24bc13b5c469cade02763acf072bd/roles/f8e128c40c034a9587c922be6a3fc432 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:22:31.862555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd438b-3ffe-4426-8c50-86350549f4f2 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:31.863153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd438b-3ffe-4426-8c50-86350549f4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:31.863153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd438b-3ffe-4426-8c50-86350549f4f2 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:31.864432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 537/1078] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:31.870114 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:31.870399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:31.870672 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:31.888740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:31.889980 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:31.962152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 542/1079] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:31.984652 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd438b-3ffe-4426-8c50-86350549f4f2 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:32.039749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:32.091527 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd438b-3ffe-4426-8c50-86350549f4f2 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:32.126734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:32.127456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:32.127730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:32.128000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:32.139389 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:32.148799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:32.150027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:32.151280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 538/1080] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:22:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:32.159347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:32.255666 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22036a8-0eee-48c9-bee3-04fb965eff6d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:32.268994 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:32.292403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:32.293078 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:32.293394 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:32.293665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:32.294411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:32.294714 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:32.296087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:32.304083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 543/1081] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:32.310102 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:32.413729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbe6f8-bf70-4d0c-abd1-23f832e5a25b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:32.424134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:32.448697 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:32.449606 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:32.449897 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:32.450168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] PATH_INFO: `/v3/users/abf63cf02f894c55ae2fa99021ee2b12` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:32.451380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] RBAC: Authorizing `identity:get_user(user_id=abf63cf02f894c55ae2fa99021ee2b12)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:32.452864 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:32.454684 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 539/1082] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:32 2022] GET /identity/v3/users/abf63cf02f894c55ae2fa99021ee2b12 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:32.460551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:32.566383 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b20b8-18c2-4939-aedc-c96ae17e8543 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:32.599211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced3342-54d6-415a-b982-b7e3adf03cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:32.600078 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced3342-54d6-415a-b982-b7e3adf03cfb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:32.600357 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced3342-54d6-415a-b982-b7e3adf03cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:32.600612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced3342-54d6-415a-b982-b7e3adf03cfb None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:32.601856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced3342-54d6-415a-b982-b7e3adf03cfb None admin] RBAC: Authorizing `identity:get_project(project_id=35baa6c59d80476f84b1a823765168d0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:32.603288 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced3342-54d6-415a-b982-b7e3adf03cfb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:32.604953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 544/1083] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:32 2022] GET /identity/v3/projects/35baa6c59d80476f84b1a823765168d0 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:32.611005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:32.715608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b56a0-0405-4acf-8940-76632a72bf8c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:32.751500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf4636-ff7a-4bfc-b950-78d30c10ad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:32.752514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf4636-ff7a-4bfc-b950-78d30c10ad5f None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:32.752806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf4636-ff7a-4bfc-b950-78d30c10ad5f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:32.753337 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf4636-ff7a-4bfc-b950-78d30c10ad5f None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0/users/abf63cf02f894c55ae2fa99021ee2b12/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:32.756149 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf4636-ff7a-4bfc-b950-78d30c10ad5f None admin] RBAC: Authorizing `identity:create_grant(project_id=35baa6c59d80476f84b1a823765168d0, user_id=abf63cf02f894c55ae2fa99021ee2b12, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:32.757722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf4636-ff7a-4bfc-b950-78d30c10ad5f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:32.773530 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 540/1084] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:22:32 2022] PUT /identity/v3/projects/35baa6c59d80476f84b1a823765168d0/users/abf63cf02f894c55ae2fa99021ee2b12/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:22:33.790872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff51f-2d15-4f66-9bba-16e0c04b4290 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:33.791466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff51f-2d15-4f66-9bba-16e0c04b4290 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:33.791466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff51f-2d15-4f66-9bba-16e0c04b4290 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:33.792773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 545/1085] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:33.798269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:33.798710 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:33.798916 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:33.816166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:33.817376 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:33.848214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:33.889953 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 541/1086] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:33.914386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ff51f-2d15-4f66-9bba-16e0c04b4290 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:33.971390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:34.031838 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ff51f-2d15-4f66-9bba-16e0c04b4290 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:34.066317 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:34.067059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.067338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.067605 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.078497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:34.086477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:34.087440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:34.088308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 546/1087] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:22:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:34.094093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:34.194001 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2cd57-e9bd-4dc0-ab51-272a5900d1b5 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:34.228857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:34.229546 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.229821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.230081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.230819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:34.231119 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:34.232486 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:34.240525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 542/1088] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:22:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:34.246328 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:34.358609 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6f014-94fe-4736-83da-d4ee6cf3bac1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:34.360162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:34.394898 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:34.395691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.395966 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.396215 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.406782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:34.408228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:34.425780 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:34.426723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:34.427643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:34.427643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:34.427643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:34.427643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:34.427643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 547/1089] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:34.432897 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:34.464666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:34.536460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3c873-5541-45e1-bcee-fb5568daa825 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:34.571723 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:34.572386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.572667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.572939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.573700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:34.574008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:34.575399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:34.587040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 543/1090] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:34.593058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:34.702773 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89871b-9bcd-40c1-90f8-4cce9fc70083 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:34.736904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:34.737741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.738017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.738292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.745738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:34.747208 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:22:34.763036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:34.764230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:34.765541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 548/1091] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:22:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:34.772109 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:34.867161 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:34.878134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0b22-6510-4282-b3bd-efcf1fe2dde7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:34.912923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:34.913693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.913967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.914292 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.915127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:34.915459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:34.916918 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c72e-d00c-4572-9136-ec55931907f9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:34.934665 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 544/1092] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:22:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:34.940550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:34.944050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb5ba-7f8b-4979-b1d6-6da4d0d3dd00 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.944340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb5ba-7f8b-4979-b1d6-6da4d0d3dd00 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.944607 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb5ba-7f8b-4979-b1d6-6da4d0d3dd00 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.946361 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 545/1093] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:34.951755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:34.952029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:34.952304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:34.957627 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:34.969485 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:34.970682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:35.037299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 546/1094] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:35.046134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f452-b80e-4b51-90d3-cf4abccbc040 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.060507 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.079708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.080355 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.080632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.080904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.139095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.148169 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.149648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.161534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30351693-a0c1-4783-9fbf-ba7153de6313 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.164411 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.194299 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.195093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.195372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.195643 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.205512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:35.213132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:35.214201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 547/1095] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:35.220281 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.245344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 549/1096] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:35.251059 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.320775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55641dda-9cdf-499e-9876-7bcaf3ed3c27 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.351166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a9f27-4141-4779-a6b1-b66231bb3832 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.355255 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.355926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.356206 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.356478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.357229 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:35.357568 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.359041 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.364134 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.365440 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.369308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 548/1097] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:35.374923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.387106 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.387914 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.388160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.388467 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.398590 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.400042 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:22:35.417667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:35.418761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:35.419703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:22:35.419703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:22:35.419703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:22:35.419703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:35.419703 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 550/1098] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:22:35.424462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.476857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0deb7-0bad-4adf-83f3-7b94a56177e1 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.509990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.510898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.511181 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.511455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] PATH_INFO: `/v3/users/abf63cf02f894c55ae2fa99021ee2b12` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.512676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] RBAC: Authorizing `identity:get_user(user_id=abf63cf02f894c55ae2fa99021ee2b12)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.514263 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.515800 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 549/1099] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/users/abf63cf02f894c55ae2fa99021ee2b12 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:35.521402 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.521759 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75add2b-0f56-46dd-9ba1-dc581bceca83 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.555522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.556304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.556585 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.556858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.557611 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:35.557918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.559287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.570699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 551/1100] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:35.576180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.585411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.602861 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.618480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f077fa-d665-4fa0-a93e-61bede2a0f47 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.655318 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.655955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.656227 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.656522 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.674815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ffe36-e648-4321-862c-2770c50935e4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.708130 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.708933 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.709207 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.709510 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.716102 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.717595 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.726136 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.727667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:22:35.731531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:35.732561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:35.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 552/1101] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:35.738697 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.796221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.826375 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:35.834782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 550/1102] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:22:35.835860 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.846301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d960-a53f-49ee-964f-d8574ec84624 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.880010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.880669 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.880949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.881227 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.882040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:35.882353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.883781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.895040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 553/1103] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:35.901676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:35.933082 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795ae77-2c40-45f2-87ad-44449fb09a07 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:35.970950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:35.971814 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:35.972084 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:35.972351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:35.973552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] RBAC: Authorizing `identity:get_project(project_id=35baa6c59d80476f84b1a823765168d0)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:35.975002 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:35.976733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 551/1104] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/projects/35baa6c59d80476f84b1a823765168d0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:35.982858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:36.002630 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a816a-d6e0-49a2-9387-4f6c6fa33882 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:36.017429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:36.035903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:36.036566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:36.036851 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:36.037123 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:36.037995 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] RBAC: Adding query filter params (role.id=f8e128c40c034a9587c922be6a3fc432, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=b4d24bc13b5c469cade02763acf072bd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:36.038340 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:36.039785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786d331-ba0b-4fd8-9cbc-fdb43e97b9c2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:36.045607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 554/1105] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/role_assignments?user.id=b4d24bc13b5c469cade02763acf072bd&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=f8e128c40c034a9587c922be6a3fc432 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:22:36.083072 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da989a5-c058-41a5-b0f0-4a31ac8afebe None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:36.121797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:36.122478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:36.122769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:36.123044 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:36.124381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=35baa6c59d80476f84b1a823765168d0, user.id=abf63cf02f894c55ae2fa99021ee2b12) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:36.124693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:36.126217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30c323-d983-49d0-81b6-e37e3a8b0ee1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:36.133095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 552/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:35 2022] GET /identity/v3/role_assignments?user.id=abf63cf02f894c55ae2fa99021ee2b12&scope.project.id=35baa6c59d80476f84b1a823765168d0&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:38.329278 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2e6b-5879-4bb9-b31b-e72a81544fcd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:38.329812 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2e6b-5879-4bb9-b31b-e72a81544fcd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:38.329812 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2e6b-5879-4bb9-b31b-e72a81544fcd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:38.331117 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 555/1107] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:38.337563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:38.338287 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:38.338479 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:38.358312 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:38.359499 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:38.438103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 553/1108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:38.463029 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2e6b-5879-4bb9-b31b-e72a81544fcd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:38.479413 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:38.576735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2e6b-5879-4bb9-b31b-e72a81544fcd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:38.619597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc830dcc-f353-403b-ba3d-37a1f8c95402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:38.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830dcc-f353-403b-ba3d-37a1f8c95402 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:38.621123 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830dcc-f353-403b-ba3d-37a1f8c95402 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:38.621493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830dcc-f353-403b-ba3d-37a1f8c95402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:38.623640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc830dcc-f353-403b-ba3d-37a1f8c95402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:38.625103 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc830dcc-f353-403b-ba3d-37a1f8c95402 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:38.626764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 556/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:38.632854 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:38.668272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:38.737487 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd6af0-09b8-40ef-8537-c63ffb02cb2b None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:38.773832 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e28f-c750-436b-b02d-dee1ce0c50fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:38.774495 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e28f-c750-436b-b02d-dee1ce0c50fc None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:38.774775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e28f-c750-436b-b02d-dee1ce0c50fc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:38.775045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e28f-c750-436b-b02d-dee1ce0c50fc None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:38.775759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e28f-c750-436b-b02d-dee1ce0c50fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:38.777214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53e28f-c750-436b-b02d-dee1ce0c50fc None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:38.803525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 554/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:22:38 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:41.031595 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572f47b-ac5e-4fef-a184-c8245dce3b1b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:41.032148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572f47b-ac5e-4fef-a184-c8245dce3b1b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:41.032148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572f47b-ac5e-4fef-a184-c8245dce3b1b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:41.033572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 557/1111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:41.039970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:41.040671 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:41.040822 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:41.062095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:41.063298 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:41.134858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 555/1112] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:41.160110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572f47b-ac5e-4fef-a184-c8245dce3b1b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:41.246275 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:41.272911 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572f47b-ac5e-4fef-a184-c8245dce3b1b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:41.311563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:41.312417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:41.312691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:41.312965 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] PATH_INFO: `/v3/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:41.314526 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] RBAC: Authorizing `identity:get_role(role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:41.315895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:41.317292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 558/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:41 2022] GET /identity/v3/roles/afd7d26260694a27a2da7aca75966a85 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:41.326442 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:41.350325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:41.442410 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309809d-5513-4a01-8776-dd06a8eec25b None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:41.476987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:41.477881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:41.478155 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:41.478421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] PATH_INFO: `/v3/users/c00a8ab1ecd840a999a326a6fada89ac` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:41.491635 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] RBAC: Authorizing `identity:get_user(user_id=c00a8ab1ecd840a999a326a6fada89ac)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:41.493105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:41.494755 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 556/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:41 2022] GET /identity/v3/users/c00a8ab1ecd840a999a326a6fada89ac => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:41.502347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:41.620979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689267bb-8d00-49a6-b54f-40b573a3cee4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:41.659298 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:41.659953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:41.660234 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:41.660491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:41.680690 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:41.738291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:41.739823 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:41.844311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 559/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:41.850415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:41.934658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:41.960541 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82a51b-99c4-4749-99a0-4858b097585b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:41.994850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994d7e-fb02-42b1-98c8-156516119dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:41.995719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994d7e-fb02-42b1-98c8-156516119dcd None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:41.995955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994d7e-fb02-42b1-98c8-156516119dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:41.996260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994d7e-fb02-42b1-98c8-156516119dcd None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:41.997585 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af994d7e-fb02-42b1-98c8-156516119dcd None admin] RBAC: Authorizing `identity:get_project(project_id=35baa6c59d80476f84b1a823765168d0)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:41.999063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af994d7e-fb02-42b1-98c8-156516119dcd None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:42.000785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 557/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:41 2022] GET /identity/v3/projects/35baa6c59d80476f84b1a823765168d0 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:42.007729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:42.058352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:42.112316 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507a279-0227-47df-9677-2f3c403f0907 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:42.147123 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:42.147761 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:42.148030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:42.148293 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:42.149118 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] RBAC: Adding query filter params (role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=35baa6c59d80476f84b1a823765168d0, user.id=c00a8ab1ecd840a999a326a6fada89ac) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:42.149455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:42.150884 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270bee6-c91a-4759-ac54-4f00ed9e735e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:42.156863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 560/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:42 2022] GET /identity/v3/role_assignments?user.id=c00a8ab1ecd840a999a326a6fada89ac&scope.project.id=35baa6c59d80476f84b1a823765168d0&role.id=afd7d26260694a27a2da7aca75966a85 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:44.303873 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57468fc1-f7b0-447d-a45e-776d775bfc48 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:44.304691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57468fc1-f7b0-447d-a45e-776d775bfc48 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:44.304691 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57468fc1-f7b0-447d-a45e-776d775bfc48 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:44.306891 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 558/1118] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:44.312406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:44.312857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:44.312948 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:44.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:44.332843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:44.402138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 561/1119] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:44.425879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57468fc1-f7b0-447d-a45e-776d775bfc48 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:44.538837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57468fc1-f7b0-447d-a45e-776d775bfc48 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:44.574263 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:44.575047 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:44.575324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:44.575591 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] PATH_INFO: `/v3/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:44.577208 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] RBAC: Authorizing `identity:get_role(role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:44.578692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:44.580069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 559/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:44 2022] GET /identity/v3/roles/afd7d26260694a27a2da7aca75966a85 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:44.586182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:44.627417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:44.694182 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c714de-4a14-40e1-a35e-9e85f5fbb2c7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:44.729160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:44.730063 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:44.730338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:44.730608 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] PATH_INFO: `/v3/users/c00a8ab1ecd840a999a326a6fada89ac` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:44.731935 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] RBAC: Authorizing `identity:get_user(user_id=c00a8ab1ecd840a999a326a6fada89ac)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:44.733494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:44.735115 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 562/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:44 2022] GET /identity/v3/users/c00a8ab1ecd840a999a326a6fada89ac => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:44.741110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:44.754762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:44.851505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90878-563e-4ecc-a6f9-1822d6855688 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:44.887352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effc803-c387-4314-be9f-9e629dd4a4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:44.888215 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effc803-c387-4314-be9f-9e629dd4a4fe None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:44.888493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effc803-c387-4314-be9f-9e629dd4a4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:44.888764 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effc803-c387-4314-be9f-9e629dd4a4fe None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:44.890093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effc803-c387-4314-be9f-9e629dd4a4fe None admin] RBAC: Authorizing `identity:get_project(project_id=35baa6c59d80476f84b1a823765168d0)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:44.891573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effc803-c387-4314-be9f-9e629dd4a4fe None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:44.893266 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 560/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:44 2022] GET /identity/v3/projects/35baa6c59d80476f84b1a823765168d0 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:44.902849 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:45.013441 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97600fbc-b2a9-4e4a-a7b1-05f40d4de560 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:45.022538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:45.054471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817066de-7005-4cd5-b51c-fc31ea08ff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:45.055389 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817066de-7005-4cd5-b51c-fc31ea08ff5f None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:45.055667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817066de-7005-4cd5-b51c-fc31ea08ff5f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:45.055942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817066de-7005-4cd5-b51c-fc31ea08ff5f None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0/users/c00a8ab1ecd840a999a326a6fada89ac/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:45.058415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817066de-7005-4cd5-b51c-fc31ea08ff5f None admin] RBAC: Authorizing `identity:create_grant(project_id=35baa6c59d80476f84b1a823765168d0, user_id=c00a8ab1ecd840a999a326a6fada89ac, role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:45.059901 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817066de-7005-4cd5-b51c-fc31ea08ff5f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:45.075579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 563/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:22:44 2022] PUT /identity/v3/projects/35baa6c59d80476f84b1a823765168d0/users/c00a8ab1ecd840a999a326a6fada89ac/roles/afd7d26260694a27a2da7aca75966a85 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:22:47.240458 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184390a-a663-4ad5-9fcf-36246bbe0fc9 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:47.241000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184390a-a663-4ad5-9fcf-36246bbe0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:47.241000 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184390a-a663-4ad5-9fcf-36246bbe0fc9 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:47.242344 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 561/1124] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:47.248764 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:47.249514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:47.249661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:47.273040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:47.274282 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:47.343587 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 564/1125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:47.378615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184390a-a663-4ad5-9fcf-36246bbe0fc9 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:47.462840 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:47.488342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184390a-a663-4ad5-9fcf-36246bbe0fc9 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:47.529732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:47.530485 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:47.530760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:47.531029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] PATH_INFO: `/v3/roles/afd7d26260694a27a2da7aca75966a85` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:47.532571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] RBAC: Authorizing `identity:get_role(role_id=afd7d26260694a27a2da7aca75966a85)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:47.533987 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:47.535395 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 562/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:47 2022] GET /identity/v3/roles/afd7d26260694a27a2da7aca75966a85 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:47.542722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:47.649667 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:47.653352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b28857-3d2b-474f-adf0-e46f72186bf6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:47.696578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:47.697475 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:47.697751 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:47.698021 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] PATH_INFO: `/v3/users/c00a8ab1ecd840a999a326a6fada89ac` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:47.699266 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] RBAC: Authorizing `identity:get_user(user_id=c00a8ab1ecd840a999a326a6fada89ac)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:47.700700 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:47.702404 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 565/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:47 2022] GET /identity/v3/users/c00a8ab1ecd840a999a326a6fada89ac => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:47.711729 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:47.827749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc97ef-54e1-47fb-ac5c-14a0c90dfc24 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:47.867506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:47.868156 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:47.868436 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:47.868707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:47.886786 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:47.945420 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:47.946891 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:48.046415 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 563/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:48.053311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:48.169519 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cb296-7af4-487d-9e5e-4516b92b4881 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:48.212776 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29626de1-d4fe-49d5-939f-8c0fc634e9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:48.213660 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29626de1-d4fe-49d5-939f-8c0fc634e9fa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:48.213939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29626de1-d4fe-49d5-939f-8c0fc634e9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:48.214214 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29626de1-d4fe-49d5-939f-8c0fc634e9fa None admin] PATH_INFO: `/v3/projects/35baa6c59d80476f84b1a823765168d0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:48.215439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29626de1-d4fe-49d5-939f-8c0fc634e9fa None admin] RBAC: Authorizing `identity:get_project(project_id=35baa6c59d80476f84b1a823765168d0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:48.216880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29626de1-d4fe-49d5-939f-8c0fc634e9fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:48.218680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 566/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:48 2022] GET /identity/v3/projects/35baa6c59d80476f84b1a823765168d0 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:48.225227 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:48.274782 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:48.329542 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654395d3-dc84-4c9f-810a-ac9fb042b520 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:48.364705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:48.365375 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:48.365660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:48.365932 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:48.366763 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] RBAC: Adding query filter params (role.id=afd7d26260694a27a2da7aca75966a85, scope.project.id=35baa6c59d80476f84b1a823765168d0, user.id=c00a8ab1ecd840a999a326a6fada89ac) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:48.367076 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:48.368528 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecf367-3e1a-474f-a6e3-e9f459f06911 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:48.374636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 564/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:48 2022] GET /identity/v3/role_assignments?user.id=c00a8ab1ecd840a999a326a6fada89ac&scope.project.id=35baa6c59d80476f84b1a823765168d0&role.id=afd7d26260694a27a2da7aca75966a85 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:50.575909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa9b3-8066-4320-904d-1929997a9947 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:50.576555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa9b3-8066-4320-904d-1929997a9947 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:50.576555 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa9b3-8066-4320-904d-1929997a9947 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:50.577901 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 567/1131] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:50.584194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:50.584618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:50.584759 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:50.606799 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:50.607996 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:50.681950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 565/1132] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:50.711683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa9b3-8066-4320-904d-1929997a9947 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:50.749372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:50.826665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa9b3-8066-4320-904d-1929997a9947 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:50.864250 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e07d5-7b1d-4b7e-b7b2-1a0985bcd0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:50.865085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e07d5-7b1d-4b7e-b7b2-1a0985bcd0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:50.865365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e07d5-7b1d-4b7e-b7b2-1a0985bcd0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:50.865673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e07d5-7b1d-4b7e-b7b2-1a0985bcd0d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:50.866843 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e07d5-7b1d-4b7e-b7b2-1a0985bcd0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:50.868292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e07d5-7b1d-4b7e-b7b2-1a0985bcd0d7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:50.869972 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 568/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:50.876109 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:50.943888 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:51.002273 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2e79c-382f-4976-a04d-b3eef4189d1a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:51.036969 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d380-849d-4c58-8d7a-333b918b50a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:51.037660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d380-849d-4c58-8d7a-333b918b50a9 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:51.037938 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d380-849d-4c58-8d7a-333b918b50a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:51.038206 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d380-849d-4c58-8d7a-333b918b50a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:51.038910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7d380-849d-4c58-8d7a-333b918b50a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:51.040360 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7d380-849d-4c58-8d7a-333b918b50a9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:51.068061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 566/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:22:50 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:53.284741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a837c7-9dce-4236-93dd-e2a6166de07f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:53.285556 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a837c7-9dce-4236-93dd-e2a6166de07f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:53.285716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a837c7-9dce-4236-93dd-e2a6166de07f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:53.287922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 569/1135] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:53.293597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:53.294046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:53.294139 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:53.313353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:53.314518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:53.381666 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 567/1136] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:53.407113 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a837c7-9dce-4236-93dd-e2a6166de07f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:53.519359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a837c7-9dce-4236-93dd-e2a6166de07f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:53.520973 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:53.555988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30551df4-b010-4906-8f85-9ba42b907b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:53.556702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30551df4-b010-4906-8f85-9ba42b907b28 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:53.556979 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30551df4-b010-4906-8f85-9ba42b907b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:53.557274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30551df4-b010-4906-8f85-9ba42b907b28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:53.558514 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30551df4-b010-4906-8f85-9ba42b907b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:53.559957 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30551df4-b010-4906-8f85-9ba42b907b28 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:53.561695 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 570/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:22:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:53.567803 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:53.594356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:53.682011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d8511-1d20-44d3-9b1b-164de67c0b60 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:53.716597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e02de6-a2c1-4bae-8280-d3f7d898816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:53.717275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e02de6-a2c1-4bae-8280-d3f7d898816f None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:53.717597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e02de6-a2c1-4bae-8280-d3f7d898816f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:53.717867 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e02de6-a2c1-4bae-8280-d3f7d898816f None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:53.718578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e02de6-a2c1-4bae-8280-d3f7d898816f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:53.720036 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e02de6-a2c1-4bae-8280-d3f7d898816f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:53.746163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 568/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:22:53 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:22:55.956546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0af140-91f7-473d-bb3d-f451181d7aef None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:55.957012 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0af140-91f7-473d-bb3d-f451181d7aef None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:55.957124 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0af140-91f7-473d-bb3d-f451181d7aef None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:55.959477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 571/1139] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:55.964855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:55.965308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:55.965400 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:55.982820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:55.983986 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:56.075869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 569/1140] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:56.100500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0af140-91f7-473d-bb3d-f451181d7aef None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:56.209968 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0af140-91f7-473d-bb3d-f451181d7aef None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:56.246824 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:56.247620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:56.247905 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:56.248192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:56.260454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:56.268277 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:56.269569 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:56.271370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 572/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:56.281789 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:56.293253 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:56.396598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba0454-7d71-4438-b994-38ba808ea5d0 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:56.436086 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:56.436785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:56.437085 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:56.437386 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:56.438146 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:56.438464 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:56.439862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:56.448645 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 570/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:56.455270 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:56.495149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:56.573160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6a107-b29e-404a-908b-317bbafa143d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:56.611561 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:56.612444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:56.612721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:56.612996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] PATH_INFO: `/v3/users/cdd3f1310ed246d9bd46b4a4503096ce` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:56.628539 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd3f1310ed246d9bd46b4a4503096ce)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:56.630052 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:56.631811 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 573/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:56 2022] GET /identity/v3/users/cdd3f1310ed246d9bd46b4a4503096ce => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:56.638461 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:56.715872 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:56.758963 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7804a-799b-48cd-9c1c-97fd80c11e49 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:56.794633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:56.795340 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:56.795619 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:56.795896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:56.865935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:56.867390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:56.940922 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:56.997285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 571/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:22:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:22:57.003777 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:57.098234 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:57.113623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa72a-5e47-4c5b-9474-92e7024b5ab5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:57.150993 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb496f-8c14-44cd-947c-a4b7fd32a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:57.151856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb496f-8c14-44cd-947c-a4b7fd32a02c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:57.152127 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb496f-8c14-44cd-947c-a4b7fd32a02c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:57.152395 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb496f-8c14-44cd-947c-a4b7fd32a02c None admin] PATH_INFO: `/v3/projects/44ec421f279b475284538fb4b9ea0dca` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:57.155783 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb496f-8c14-44cd-947c-a4b7fd32a02c None admin] RBAC: Authorizing `identity:get_project(project_id=44ec421f279b475284538fb4b9ea0dca)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:57.157202 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb496f-8c14-44cd-947c-a4b7fd32a02c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:57.158950 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 574/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:22:57 2022] GET /identity/v3/projects/44ec421f279b475284538fb4b9ea0dca => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:57.165506 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:57.269118 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8d311-3922-4198-81ee-20fea04c0c52 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:57.304120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:57.304775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:57.305054 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:57.305353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:57.306173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=44ec421f279b475284538fb4b9ea0dca, user.id=cdd3f1310ed246d9bd46b4a4503096ce) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:57.306481 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:57.307940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dc5e9-0cee-41aa-92c4-bfcfc2857ac8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:57.313732 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 572/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:22:57 2022] GET /identity/v3/role_assignments?user.id=cdd3f1310ed246d9bd46b4a4503096ce&scope.project.id=44ec421f279b475284538fb4b9ea0dca&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:59.502807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786a1b0-a0ed-479d-9264-89148947cc8f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:59.503732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786a1b0-a0ed-479d-9264-89148947cc8f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:59.503732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786a1b0-a0ed-479d-9264-89148947cc8f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:59.505860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 575/1147] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:22:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:22:59.512312 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:59.513038 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:59.513220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:59.536857 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:22:59.538077 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:22:59.564636 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:22:59.607724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 573/1148] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:22:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:22:59.630564 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786a1b0-a0ed-479d-9264-89148947cc8f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:59.734613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786a1b0-a0ed-479d-9264-89148947cc8f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:59.769494 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:59.770199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:59.770472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:59.770740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:59.782406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:22:59.789881 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:22:59.790976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:22:59.791999 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 576/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:22:59.797269 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:59.905407 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ace16b-129f-470a-b9c1-c2331c22a7a1 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:22:59.943644 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:22:59.944631 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:22:59.945007 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:22:59.945410 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:22:59.946575 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:22:59.946978 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:22:59.949121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:22:59.958536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 574/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:22:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:22:59.964631 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:22:59.983958 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:00.079808 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716541b9-10b8-47e0-9c6d-5167afa35437 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:00.114487 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:00.115352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:00.115620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:00.115885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] PATH_INFO: `/v3/users/cdd3f1310ed246d9bd46b4a4503096ce` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:00.117181 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd3f1310ed246d9bd46b4a4503096ce)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:00.118641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:00.120258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 577/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:22:59 2022] GET /identity/v3/users/cdd3f1310ed246d9bd46b4a4503096ce => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:00.126854 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:00.166935 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:00.239726 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03559c17-94bf-4c86-aefc-c904488cd792 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:00.274025 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43065380-2549-4981-8505-6ccc0793851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:00.274850 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43065380-2549-4981-8505-6ccc0793851c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:00.275121 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43065380-2549-4981-8505-6ccc0793851c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:00.275390 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43065380-2549-4981-8505-6ccc0793851c None admin] PATH_INFO: `/v3/projects/44ec421f279b475284538fb4b9ea0dca` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:00.276623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43065380-2549-4981-8505-6ccc0793851c None admin] RBAC: Authorizing `identity:get_project(project_id=44ec421f279b475284538fb4b9ea0dca)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:00.278103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43065380-2549-4981-8505-6ccc0793851c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:00.280214 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 575/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:23:00 2022] GET /identity/v3/projects/44ec421f279b475284538fb4b9ea0dca => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:00.286304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:00.385789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:00.402788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac035-2449-4d07-89f5-689d1ea0a059 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:00.437620 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee44d52-641a-45c4-8f2c-d7e53a23b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:00.438495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee44d52-641a-45c4-8f2c-d7e53a23b596 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:00.438766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee44d52-641a-45c4-8f2c-d7e53a23b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:00.439032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee44d52-641a-45c4-8f2c-d7e53a23b596 None admin] PATH_INFO: `/v3/projects/44ec421f279b475284538fb4b9ea0dca/users/cdd3f1310ed246d9bd46b4a4503096ce/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:00.441225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee44d52-641a-45c4-8f2c-d7e53a23b596 None admin] RBAC: Authorizing `identity:create_grant(project_id=44ec421f279b475284538fb4b9ea0dca, user_id=cdd3f1310ed246d9bd46b4a4503096ce, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:00.442684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee44d52-641a-45c4-8f2c-d7e53a23b596 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:00.457772 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 578/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:23:00 2022] PUT /identity/v3/projects/44ec421f279b475284538fb4b9ea0dca/users/cdd3f1310ed246d9bd46b4a4503096ce/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:02.662003 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb36ce0-4fe7-44a9-be4d-6e6e0adf0dfa None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:02.662944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb36ce0-4fe7-44a9-be4d-6e6e0adf0dfa None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:02.662944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb36ce0-4fe7-44a9-be4d-6e6e0adf0dfa None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:02.665040 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 576/1154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:02.673312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:02.673768 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:02.673891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:02.701257 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:02.702461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:02.772647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 579/1155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:02.805262 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb36ce0-4fe7-44a9-be4d-6e6e0adf0dfa None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:02.917228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:02.925751 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb36ce0-4fe7-44a9-be4d-6e6e0adf0dfa None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:02.962118 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:02.962947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:02.963226 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:02.963493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:02.974157 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:02.981750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:02.982959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:23:02.984272 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 577/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:23:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:02.991051 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:03.078251 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:03.108406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a463fda-26ad-4b15-9c43-0fec24726130 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:03.185038 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:03.185648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:03.185934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:03.186186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:03.186938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:03.187244 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:03.188635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:03.198276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 580/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:23:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:03.204998 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:03.317254 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d8f97-c2c6-4b1b-b488-9437010662c5 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:03.356280 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:03.357193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:03.357509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:03.357779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] PATH_INFO: `/v3/users/cdd3f1310ed246d9bd46b4a4503096ce` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:03.359010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd3f1310ed246d9bd46b4a4503096ce)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:03.360532 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:03.362497 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 578/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:23:03 2022] GET /identity/v3/users/cdd3f1310ed246d9bd46b4a4503096ce => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:03.368649 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:03.480312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c996b2-e3ef-4089-91a0-8a651a60ded1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:03.516821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:03.517495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:03.517776 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:03.518044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:03.521833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:03.599679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:03.601106 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:03.710862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 581/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:23:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:23:03.720080 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:03.750196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:03.834228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1fa92-a7dd-478f-89a5-dc10bbb0a110 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:03.874350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a290c52-85fc-4881-bd24-d4fb6fc174c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:03.875221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a290c52-85fc-4881-bd24-d4fb6fc174c3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:03.875598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a290c52-85fc-4881-bd24-d4fb6fc174c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:03.875869 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a290c52-85fc-4881-bd24-d4fb6fc174c3 None admin] PATH_INFO: `/v3/projects/44ec421f279b475284538fb4b9ea0dca` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:03.877189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a290c52-85fc-4881-bd24-d4fb6fc174c3 None admin] RBAC: Authorizing `identity:get_project(project_id=44ec421f279b475284538fb4b9ea0dca)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:03.878689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a290c52-85fc-4881-bd24-d4fb6fc174c3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:03.880441 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 579/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:23:03 2022] GET /identity/v3/projects/44ec421f279b475284538fb4b9ea0dca => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:03.886912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:03.922708 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:04.001218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9aa59-f2b1-4ed0-a9fb-eb134418427d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:04.039706 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:04.040359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:04.040647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:04.040918 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:04.041785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=44ec421f279b475284538fb4b9ea0dca, user.id=cdd3f1310ed246d9bd46b4a4503096ce) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:04.042102 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:04.043566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46359f-58f3-4c22-8462-276b29fe45e2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:04.049737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 582/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:23:03 2022] GET /identity/v3/role_assignments?user.id=cdd3f1310ed246d9bd46b4a4503096ce&scope.project.id=44ec421f279b475284538fb4b9ea0dca&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:06.289293 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f793-474f-4794-9961-2b50f8a30b8a None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:06.290174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f793-474f-4794-9961-2b50f8a30b8a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:06.290174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f793-474f-4794-9961-2b50f8a30b8a None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:06.292259 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 580/1162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:06.298839 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:06.299550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:06.299727 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:06.321418 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:06.322597 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:06.391641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 583/1163] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:06.419562 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9f793-474f-4794-9961-2b50f8a30b8a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:06.516859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:06.531476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9f793-474f-4794-9961-2b50f8a30b8a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:06.578344 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:06.578926 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:06.579207 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:06.579482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:06.648859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:06.650356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:06.743933 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:06.760261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 581/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:23:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:23:06.766551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:06.797454 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:06.866560 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9622-5dba-433c-b3a1-8d76d72189f0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:06.902079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:06.902872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:06.903154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:06.903424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:06.910825 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:06.912284 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ae08a71c-eb0f-46e0-ac72-69894aad3c44 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:06.926792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:06.927998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:23:06.928909 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 584/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:23:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:06.934260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:07.047976 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a726e-3fcb-4cc8-b7e3-995256a8a227 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:07.089200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:07.089894 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:07.090168 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:07.090439 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:07.091163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:07.091573 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:07.092984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1292-ab16-49a2-b4e7-d3fe5f5f4a9f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:07.100424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 582/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:23:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:09.256402 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5520-208d-4689-b124-a3e8638c4441 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:09.257281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5520-208d-4689-b124-a3e8638c4441 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:09.257281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5520-208d-4689-b124-a3e8638c4441 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:09.259415 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 585/1167] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:09.264862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb17b32-6ac3-476f-b2a3-22b276a8099d None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:09.265331 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb17b32-6ac3-476f-b2a3-22b276a8099d None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:09.265432 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb17b32-6ac3-476f-b2a3-22b276a8099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:09.284601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-ecb17b32-6ac3-476f-b2a3-22b276a8099d None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:09.285805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ecb17b32-6ac3-476f-b2a3-22b276a8099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:09.335250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:09.362749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 583/1168] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:09.385472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e5520-208d-4689-b124-a3e8638c4441 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:09.486006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:09.498307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e5520-208d-4689-b124-a3e8638c4441 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:09.533802 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ad793-ca27-407d-8638-cf99456b4087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:09.534382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ad793-ca27-407d-8638-cf99456b4087 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:09.534658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ad793-ca27-407d-8638-cf99456b4087 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:09.534932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ad793-ca27-407d-8638-cf99456b4087 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:09.535678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ad793-ca27-407d-8638-cf99456b4087 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:09.537056 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ad793-ca27-407d-8638-cf99456b4087 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:09.553985 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 586/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Thu May 19 14:23:09 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:11.780231 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe0b3-9a0e-4104-b474-361d1a776ca6 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.781137 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe0b3-9a0e-4104-b474-361d1a776ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.781137 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe0b3-9a0e-4104-b474-361d1a776ca6 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.783313 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 584/1170] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:11.789618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:11.790068 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:11.790229 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:11.809518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:11.810725 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:11.889360 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 587/1171] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:11.913462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fe0b3-9a0e-4104-b474-361d1a776ca6 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.018847 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fe0b3-9a0e-4104-b474-361d1a776ca6 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.057943 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2438de-a5e7-412c-9dd8-152b60a75fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.058736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2438de-a5e7-412c-9dd8-152b60a75fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.059015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2438de-a5e7-412c-9dd8-152b60a75fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.059289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2438de-a5e7-412c-9dd8-152b60a75fa8 None admin] PATH_INFO: `/v3/domains/1c6189c86ede43079ee4c8de0557d133` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.069817 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2438de-a5e7-412c-9dd8-152b60a75fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1c6189c86ede43079ee4c8de0557d133)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.071283 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2438de-a5e7-412c-9dd8-152b60a75fa8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.072901 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 585/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 14:23:11 2022] GET /identity/v3/domains/1c6189c86ede43079ee4c8de0557d133 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:12.082189 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:12.168685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:12.188922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35942311-9c6a-46e7-af6f-0bee4b0f476f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:12.223485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fb971-de55-4dac-afec-8aa51336181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:12.224133 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fb971-de55-4dac-afec-8aa51336181f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:12.224412 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fb971-de55-4dac-afec-8aa51336181f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:12.224681 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fb971-de55-4dac-afec-8aa51336181f None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:12.225419 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fb971-de55-4dac-afec-8aa51336181f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:12.226859 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fb971-de55-4dac-afec-8aa51336181f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:12.254880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 588/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:23:12 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:14.498383 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272063e3-c9a7-488d-b422-076278a2a846 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:14.498993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272063e3-c9a7-488d-b422-076278a2a846 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:14.498993 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272063e3-c9a7-488d-b422-076278a2a846 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:14.500335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 586/1174] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:14.506814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:14.507537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:14.507673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:14.533832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:14.535018 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:14.603635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 589/1175] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:14.628369 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272063e3-c9a7-488d-b422-076278a2a846 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:14.685146 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:14.740518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272063e3-c9a7-488d-b422-076278a2a846 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:14.775417 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9e3f2-db17-40b1-96d9-1fde9b3dc88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:14.776212 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e3f2-db17-40b1-96d9-1fde9b3dc88c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:14.776487 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e3f2-db17-40b1-96d9-1fde9b3dc88c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:14.776756 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e3f2-db17-40b1-96d9-1fde9b3dc88c None admin] PATH_INFO: `/v3/domains/1c6189c86ede43079ee4c8de0557d133` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:14.778046 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e3f2-db17-40b1-96d9-1fde9b3dc88c None admin] RBAC: Authorizing `identity:get_domain(domain_id=1c6189c86ede43079ee4c8de0557d133)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:14.779845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e3f2-db17-40b1-96d9-1fde9b3dc88c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:14.781490 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 587/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 14:23:14 2022] GET /identity/v3/domains/1c6189c86ede43079ee4c8de0557d133 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:14.787422 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:14.851990 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:14.900035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33c366-22ae-4734-84a2-bc71b8cea861 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:14.936754 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30796c1-c252-4641-a091-1d59c6110948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:14.937467 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30796c1-c252-4641-a091-1d59c6110948 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:14.937743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30796c1-c252-4641-a091-1d59c6110948 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:14.938013 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30796c1-c252-4641-a091-1d59c6110948 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:14.938732 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30796c1-c252-4641-a091-1d59c6110948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:14.940176 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30796c1-c252-4641-a091-1d59c6110948 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:14.963426 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 590/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:23:14 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:23:17.215460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aebf59-d4e2-4eac-ba18-35f9f59e8699 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:17.216315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aebf59-d4e2-4eac-ba18-35f9f59e8699 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:17.216315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aebf59-d4e2-4eac-ba18-35f9f59e8699 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:17.218518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 588/1178] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:17.224743 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:17.225171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:17.225371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:17.245336 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:17.246529 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:17.315758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 591/1179] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:17.343106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aebf59-d4e2-4eac-ba18-35f9f59e8699 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:17.459657 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aebf59-d4e2-4eac-ba18-35f9f59e8699 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:17.472779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:17.494883 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:17.495694 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:17.495970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:17.496241 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:17.507087 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:17.514716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:17.515863 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:23:17.516793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 589/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:17.521998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:17.577731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:17.630324 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d2b79-5c11-4ea0-9f06-667e3518c971 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:17.667544 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:17.668238 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:17.668517 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:17.668789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:17.669568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:17.669878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:17.671378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:17.679417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 592/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:17.685439 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:17.785653 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ce0e7-c8e3-49c8-ad25-35de9eff68ba None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:17.820525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:17.821399 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:17.821674 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:17.821939 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] PATH_INFO: `/v3/users/9f492852934d4cf5a57cef1f5e8c94c4` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:17.835663 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] RBAC: Authorizing `identity:get_user(user_id=9f492852934d4cf5a57cef1f5e8c94c4)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:17.837093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:17.838734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 590/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:23:17 2022] GET /identity/v3/users/9f492852934d4cf5a57cef1f5e8c94c4 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:17.847465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:17.960678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05472-93eb-48b7-bea2-18fdc14f8e48 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:17.971226 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:18.001351 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:18.002014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:18.002292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:18.002562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:18.071642 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:18.073017 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:18.172647 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 593/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:23:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:23:18.181558 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:18.221454 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:18.294985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c05eb0-1f6c-4d10-9fcc-b03177b3a83f None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:18.329760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda8e48-c4f0-41f1-9d3b-d7100d67f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:18.330540 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda8e48-c4f0-41f1-9d3b-d7100d67f149 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:18.330812 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda8e48-c4f0-41f1-9d3b-d7100d67f149 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:18.331078 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda8e48-c4f0-41f1-9d3b-d7100d67f149 None admin] PATH_INFO: `/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:18.332293 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda8e48-c4f0-41f1-9d3b-d7100d67f149 None admin] RBAC: Authorizing `identity:get_project(project_id=422d9c06c8004a9789cab8ef9ac64d3a)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:18.333769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda8e48-c4f0-41f1-9d3b-d7100d67f149 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:18.335451 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 591/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:23:18 2022] GET /identity/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:18.341951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:18.358571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:18.454041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fe6de-12f6-49ae-afb5-c927733ad21f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:18.489287 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:18.489953 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:18.490226 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:18.490456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:18.491306 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=422d9c06c8004a9789cab8ef9ac64d3a, user.id=9f492852934d4cf5a57cef1f5e8c94c4) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:18.491640 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:18.493067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133495-25d9-405e-9bed-084e5dc1f5c7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:18.498696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 594/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:23:18 2022] GET /identity/v3/role_assignments?user.id=9f492852934d4cf5a57cef1f5e8c94c4&scope.project.id=422d9c06c8004a9789cab8ef9ac64d3a&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:23:20.651108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397aa98-432d-4601-8cd0-4f25cfaa7f33 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:20.651695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397aa98-432d-4601-8cd0-4f25cfaa7f33 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:20.651695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397aa98-432d-4601-8cd0-4f25cfaa7f33 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:20.653010 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 592/1186] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:20.658873 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:20.659329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:20.659423 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:20.677131 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:20.678329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:20.767694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 595/1187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:20.790634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397aa98-432d-4601-8cd0-4f25cfaa7f33 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:20.898175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:20.903222 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397aa98-432d-4601-8cd0-4f25cfaa7f33 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:20.939736 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:20.940532 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:20.940806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:20.941071 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:20.950984 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:20.958689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:20.959771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:23:20.960825 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 593/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:20.966648 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:20.971318 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:21.075912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be45cc5-7f43-45d7-a692-ceec7c4da4a6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:21.115792 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:21.116460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:21.116738 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:21.117010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:21.117786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:21.118089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:21.119433 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:21.128014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 596/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:21.135635 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:21.249181 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f19363-181b-40e4-adb1-6a488eecc04b None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:21.283375 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:21.284238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:21.284511 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:21.284819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] PATH_INFO: `/v3/users/9f492852934d4cf5a57cef1f5e8c94c4` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:21.286059 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] RBAC: Authorizing `identity:get_user(user_id=9f492852934d4cf5a57cef1f5e8c94c4)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:21.287493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:21.289095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 594/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:23:21 2022] GET /identity/v3/users/9f492852934d4cf5a57cef1f5e8c94c4 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:21.295834 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:21.366376 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:21.416112 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413cc51-5dcc-422c-902b-a1c54130665f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:21.455146 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ed0e-483c-4393-9738-1e0afdab25f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:21.456005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ed0e-483c-4393-9738-1e0afdab25f4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:21.456288 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ed0e-483c-4393-9738-1e0afdab25f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:21.456556 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ed0e-483c-4393-9738-1e0afdab25f4 None admin] PATH_INFO: `/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:21.457895 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ed0e-483c-4393-9738-1e0afdab25f4 None admin] RBAC: Authorizing `identity:get_project(project_id=422d9c06c8004a9789cab8ef9ac64d3a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:21.459331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ed0e-483c-4393-9738-1e0afdab25f4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:21.461049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 597/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:23:21 2022] GET /identity/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:21.467601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:21.499162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:21.579403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ab3d2-34c9-49ca-bafe-b064f62df374 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:21.622150 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b9454-59b3-4e4c-9a96-2c254d2efd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:21.623029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9454-59b3-4e4c-9a96-2c254d2efd5f None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:21.623309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9454-59b3-4e4c-9a96-2c254d2efd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:21.623584 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9454-59b3-4e4c-9a96-2c254d2efd5f None admin] PATH_INFO: `/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a/users/9f492852934d4cf5a57cef1f5e8c94c4/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:21.625724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9454-59b3-4e4c-9a96-2c254d2efd5f None admin] RBAC: Authorizing `identity:create_grant(project_id=422d9c06c8004a9789cab8ef9ac64d3a, user_id=9f492852934d4cf5a57cef1f5e8c94c4, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:21.627171 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9454-59b3-4e4c-9a96-2c254d2efd5f None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:21.642797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 595/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:23:21 2022] PUT /identity/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a/users/9f492852934d4cf5a57cef1f5e8c94c4/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:23:23.796862 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ea57e-09a1-4f00-8e87-f03577f64d3b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.797497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ea57e-09a1-4f00-8e87-f03577f64d3b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.797497 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ea57e-09a1-4f00-8e87-f03577f64d3b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.798842 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 598/1193] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:23:23.805294 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:23.805980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:23.806239 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:23.831859 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:23:23.833063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:23:23.913049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 596/1194] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:23:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:23:23.936500 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ea57e-09a1-4f00-8e87-f03577f64d3b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.048167 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ea57e-09a1-4f00-8e87-f03577f64d3b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.085748 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.086522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.086804 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.087075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.089371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.100010 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:23:24.107541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:23:24.108607 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:23:24.109778 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 599/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:23:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:23:24.115433 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.136037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.226063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5228b0-2e08-42c4-83fa-5374c521305e None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.263579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.264238 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.264521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.264852 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.265637 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:24.265947 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.267341 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.275730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 597/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.284869 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.388391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452bd7-3c21-41b4-9e93-c574d43b05d4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.426315 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.427175 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.427445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.427712 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] PATH_INFO: `/v3/users/9f492852934d4cf5a57cef1f5e8c94c4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.428921 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] RBAC: Authorizing `identity:get_user(user_id=9f492852934d4cf5a57cef1f5e8c94c4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.430373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.431984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 600/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/users/9f492852934d4cf5a57cef1f5e8c94c4 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.441352 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.527913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.553740 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91ad14-9de3-4b68-8e6d-80d906e06309 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.587940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.588578 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.588856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.589124 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.657862 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.659428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.740141 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.772370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 598/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:23:24.778491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:24.842847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:24.884100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297ccd-da08-4d6b-940e-bd59b708ae01 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:24.921858 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88688f-5171-41da-94e1-7bfe5943a733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:24.922626 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88688f-5171-41da-94e1-7bfe5943a733 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:24.922903 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88688f-5171-41da-94e1-7bfe5943a733 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:24.923166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88688f-5171-41da-94e1-7bfe5943a733 None admin] PATH_INFO: `/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:24.924421 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88688f-5171-41da-94e1-7bfe5943a733 None admin] RBAC: Authorizing `identity:get_project(project_id=422d9c06c8004a9789cab8ef9ac64d3a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:24.925876 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88688f-5171-41da-94e1-7bfe5943a733 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:24.927550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 601/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/projects/422d9c06c8004a9789cab8ef9ac64d3a => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:24.934324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:23:25.033837 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f496-d65c-4a39-994f-21d94c128ee0 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:23:25.068923 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:23:25.069602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:23:25.069881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:23:25.070150 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:23:25.070977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=422d9c06c8004a9789cab8ef9ac64d3a, user.id=9f492852934d4cf5a57cef1f5e8c94c4) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:23:25.071287 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:23:25.072731 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077d9ef-22b7-4429-b5cb-ed04a358b415 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:23:25.078748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 599/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:23:24 2022] GET /identity/v3/role_assignments?user.id=9f492852934d4cf5a57cef1f5e8c94c4&scope.project.id=422d9c06c8004a9789cab8ef9ac64d3a&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:23:34.916614 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=35 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:23:35.075912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=79 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.419285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633344c0-a7c8-41b0-9dfb-c63cb061bbec None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.419913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633344c0-a7c8-41b0-9dfb-c63cb061bbec None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.419913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633344c0-a7c8-41b0-9dfb-c63cb061bbec None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.421653 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 602/1201] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:25.427954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.428474 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.428574 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.447434 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:25.448688 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:25.540460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 600/1202] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:25.562680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633344c0-a7c8-41b0-9dfb-c63cb061bbec None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.696434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633344c0-a7c8-41b0-9dfb-c63cb061bbec None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.740427 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.741460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.741741 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.742014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.755391 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.757069 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:25.783370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:25.784579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:25.785623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:25.785623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:25.785623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:25.785623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:25.785623 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 603/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:25.793846 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:25.912615 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92d007-c150-49ad-b25b-4709409a017b None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:25.947325 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:25.954024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:25.954820 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:25.955306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:25.955534 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:25.956108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:25.956387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:25.957802 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e667e-1425-43bc-89ec-99984acc97aa None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:25.972836 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 601/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:25.980331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:26.039855 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:26.107377 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526cd1a-ae43-4e64-a476-b83379499aa3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:26.152645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910780c9-07a2-49ce-8ca3-f5146fc24e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:26.153661 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910780c9-07a2-49ce-8ca3-f5146fc24e6a None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:26.153994 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910780c9-07a2-49ce-8ca3-f5146fc24e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:26.154274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910780c9-07a2-49ce-8ca3-f5146fc24e6a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:26.154922 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910780c9-07a2-49ce-8ca3-f5146fc24e6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:26.156347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910780c9-07a2-49ce-8ca3-f5146fc24e6a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:26.191004 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 604/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:24:25 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:24:28.465510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568683d-e1cc-4ffa-8e0b-16491bd2b5a3 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.466103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568683d-e1cc-4ffa-8e0b-16491bd2b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.466103 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568683d-e1cc-4ffa-8e0b-16491bd2b5a3 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.467444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 602/1206] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:28.473016 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.473550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.473550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.490947 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:28.492323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:28.561211 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 605/1207] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:28.584632 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568683d-e1cc-4ffa-8e0b-16491bd2b5a3 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.703513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568683d-e1cc-4ffa-8e0b-16491bd2b5a3 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.742826 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.743593 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.743874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.744148 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.757940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:28.765774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:28.766879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:28.772770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 603/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:28.773930 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:28.820434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:28.890135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d203-8260-4d13-a9e3-72eb4528a5cc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:28.928602 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:28.929360 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:28.929638 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:28.929907 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:28.930707 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:28.931006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:28.934403 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:28.945286 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 606/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:28.948599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.003419 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.058791 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b02d36-0ee0-4968-89ed-564adc2dd53e None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.097950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.098792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.099081 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.099338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.111190 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.112719 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:29.133309 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.134324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.135686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:29.135686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:29.135686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:29.135686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:29.135686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 604/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.140386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.257739 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.260766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e37d4-aee1-4d70-89c5-81b041ea27c0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.301274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.301274 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.301645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.301848 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.302596 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:29.302888 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.304306 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.321260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 607/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:29.326781 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.447359 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650e57-3625-4c04-ad0a-e40463416a02 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.448974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.488066 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.488920 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.489364 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.489692 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.498702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.500173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:29.519392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:29.520677 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:29.521780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 605/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:29.526339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.637818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83b363-6582-432a-9cca-6a2b1346171d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.675781 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.676461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.676747 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.677008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.677913 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:29.678279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.679659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:29.697728 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 608/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:29.703676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:29.811946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f86e1-6a23-4fc4-8d91-e821db06ecd6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:29.852944 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:29.853634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:29.854722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:29.854722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:29.883795 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:29.924571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:29.926193 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.043470 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 606/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:24:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:24:30.049262 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.072417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.178880 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981eccdb-e7ee-4932-904a-d479a52ae5b1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.225412 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.226235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.226511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.226782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.237954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.239373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:30.259228 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.260460 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.266339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:30.266339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:30.266339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:30.266339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:30.266339 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 609/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.267650 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.333505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.378769 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9199e5-d3a2-4b5e-88b4-b00d11a91fd8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.419446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.421659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.421659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.421659 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.421957 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.422052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.423678 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.437685 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 607/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:30.443159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.517532 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.554515 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4efd-f314-47d9-a7c6-36b51dd9ee26 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.592386 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.593158 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.593490 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.593756 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.601014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.602496 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:30.617058 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:30.618379 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:30.619302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:30.623165 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 610/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:30.624446 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.745489 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48943c-f34e-4c3f-8471-be34a34e9f5a None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.777355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.790936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.791705 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.791985 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.792265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.793170 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.793561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.795109 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb04d48-364f-4b7b-b37e-2c35fc2033ab None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.811946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 608/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:30.818279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:30.933718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09946461-0f1c-408b-84fe-92952b7e3fb0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:30.966886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:30.974307 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:30.975109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:30.975461 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:30.975729 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:30.976759 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=a97c1ece267e4ca8b01bf7436123843d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:30.977111 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:30.978670 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ba106-87ed-4cf2-90d6-160032c63e76 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:30.988444 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 611/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:24:30 2022] GET /identity/v3/role_assignments?user.id=a97c1ece267e4ca8b01bf7436123843d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:33.176227 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956885b1-db13-48f0-b6a0-5f653291c7fc None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.176783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956885b1-db13-48f0-b6a0-5f653291c7fc None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.176783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956885b1-db13-48f0-b6a0-5f653291c7fc None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.178362 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 609/1220] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:24:33.183632 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011449-859a-4453-943e-b17137d6c685 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.184087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011449-859a-4453-943e-b17137d6c685 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.184192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011449-859a-4453-943e-b17137d6c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.201683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d3011449-859a-4453-943e-b17137d6c685 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:33.203099 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d3011449-859a-4453-943e-b17137d6c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:33.292856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 612/1221] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:33.315927 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956885b1-db13-48f0-b6a0-5f653291c7fc None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.436117 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956885b1-db13-48f0-b6a0-5f653291c7fc None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.470703 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.471421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.471697 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.471962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.482545 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:33.490270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:33.491333 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:33.492220 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 610/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:33.497685 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011449-859a-4453-943e-b17137d6c685 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.608711 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011449-859a-4453-943e-b17137d6c685 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.632633 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.647306 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.647949 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.648230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.648557 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.649289 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:33.649599 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.650975 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.659910 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 613/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:33.665896 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.712228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:33.776443 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb17e43-182f-4a19-b3b3-fa34c0526661 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:33.810203 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:33.810975 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:33.811261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:33.811529 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:33.822910 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:33.824324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:33.842994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:33.844018 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:33.845090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:33.845090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:33.845090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:33.845090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:33.845090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 611/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:33.850209 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:33.976331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431c5f-40df-48f0-b0be-484e6541e141 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.017346 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.018162 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.018439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.018717 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.019638 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:34.019940 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.021455 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.034465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 614/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:34.042185 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.140569 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.148704 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1b9a5-9700-4d44-bfd9-5d9b04e8c706 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.189460 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.190262 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.190513 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.190773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.199444 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.201045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:34.217806 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:34.219060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:34.220216 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 612/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:34.225545 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.279280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.335049 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b9c6d-8d18-4bb4-8e10-559b30015e21 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.370588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.371291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.371567 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.371837 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.372695 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:34.372996 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.374593 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.392533 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 615/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:34.398308 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.512758 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca05b-0f68-487c-b67f-447f1140bbbb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.546175 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.547153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.547387 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.547656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.564328 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.566713 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:34.585959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:34.587335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:34.588317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:34.588317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:34.588317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:34.588317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:34.588317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 613/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:34.594323 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.729652 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2e67-d281-4eef-94e6-504ef8639f8a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.732252 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.768225 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.769150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.769451 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.769833 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.770528 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:34.770828 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.772237 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.785856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 616/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:34.792306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:34.804903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:34.910892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048838-5540-45ff-9ffd-ddd6ab7b7539 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:34.945709 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:34.946698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:34.947065 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:34.947345 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:34.959403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:34.960980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:34.976014 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:34.977267 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:34.978561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 614/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:34.983641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.090205 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec41c80-dfe6-4823-9d31-20374f41574c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.124143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.124824 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.125097 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.125399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.126210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:35.126530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.127984 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a80030-43e6-45a2-a126-53e39e44854d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.143786 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 617/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:35.149886 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:35.223265 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:35.266363 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82441cbc-b674-4b02-aa00-ec8a2a866ce6 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:35.305681 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7790f-5407-4cce-9e6a-2c461dc5a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:35.306693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7790f-5407-4cce-9e6a-2c461dc5a7e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:35.307037 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7790f-5407-4cce-9e6a-2c461dc5a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:35.307315 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7790f-5407-4cce-9e6a-2c461dc5a7e8 None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/a97c1ece267e4ca8b01bf7436123843d/roles/0f35caa7049f454eb6e09cd12fe8a8c2` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:35.331127 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7790f-5407-4cce-9e6a-2c461dc5a7e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=a97c1ece267e4ca8b01bf7436123843d, role_id=0f35caa7049f454eb6e09cd12fe8a8c2)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:35.332849 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7790f-5407-4cce-9e6a-2c461dc5a7e8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:35.347170 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 615/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:24:35 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/a97c1ece267e4ca8b01bf7436123843d/roles/0f35caa7049f454eb6e09cd12fe8a8c2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:24:37.586848 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc2f45-8990-4319-9a42-b716b2b023c0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.587453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc2f45-8990-4319-9a42-b716b2b023c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.587453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc2f45-8990-4319-9a42-b716b2b023c0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.589192 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 618/1233] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:37.594323 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.594891 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.594991 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.612999 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:37.614406 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:37.700094 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 616/1234] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:37.725976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc2f45-8990-4319-9a42-b716b2b023c0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.755178 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:37.838292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc2f45-8990-4319-9a42-b716b2b023c0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:37.876952 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:37.877938 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:37.878224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:37.878491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:37.891032 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:37.899718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:37.900758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:37.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 619/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:37.907813 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:37.956159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.011532 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6de81-61df-4354-9257-bb90b3bf5255 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.051913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.052651 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.052936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.053275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.054011 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.054320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.055709 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.064520 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 617/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:38.070799 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.169470 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.187807 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72c4d4-e3f4-4016-8981-3c6b024a467a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.225566 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.226337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.226616 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.226886 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.238758 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.241137 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:38.261889 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:38.263037 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:38.268691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:38.268691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:38.268691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:38.268691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:38.268691 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 620/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:38.269050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.379365 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc40fd-2bdc-4d23-b2f4-f740024321a7 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.390380 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.416773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.417468 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.417750 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.418022 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.418747 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.419053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.420480 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.432494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 618/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:38.438397 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.572370 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde4f8e-acbc-4c38-afa8-a227ed89f655 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.611646 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.612577 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.612936 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.613294 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.617696 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:38.624235 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.626168 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:38.643138 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:38.644382 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:38.645442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 621/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:38.651567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.775442 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51770593-0527-4582-b325-8e2301cf5b67 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:38.814159 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:38.814844 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:38.815128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:38.815424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:38.816268 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:38.816579 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:38.818069 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:38.840976 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 619/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:38.847319 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:38.962981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198b564-7c1d-4ff5-9c43-80d451d1f4d2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.003805 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.004586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.004900 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.005186 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.074875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.086067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.087635 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.194055 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 622/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:24:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:24:39.199878 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.230182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.318834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141145d6-4fb7-4229-851a-678d6db95242 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.368015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.369463 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.369956 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.370431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.386707 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.388913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:39.410250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:39.411368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:39.412473 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:39.412473 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:39.412473 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:39.412473 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:39.412473 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 620/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:39.418125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.522160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.529150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7eac-f85a-4f44-a7d6-2eae46def676 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.569073 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.569831 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.570223 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.570595 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.571465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:39.571815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.573537 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.586981 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 623/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:39.593195 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.687186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:39.711608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d8883-1816-46f0-a0ca-bf45c22723bc None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.748792 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.749627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.749908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.750179 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.757042 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.758602 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:39.774510 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:39.775494 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:39.776471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 621/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:39.782482 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:39.892874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08434732-feae-4ab1-a533-94dbfe5a4b30 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:39.929024 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:39.929731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:39.930014 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:39.930280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:39.931079 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:39.931393 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:39.932827 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb279a-4de8-4639-a341-9aa70a129189 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:39.949766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 624/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:39.955879 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:40.067228 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cbe9-2427-443b-ae92-56d2c1c75e7c None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:40.111368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:40.112052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:40.112356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:40.112616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:40.113501 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] RBAC: Adding query filter params (role.id=0f35caa7049f454eb6e09cd12fe8a8c2, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=a97c1ece267e4ca8b01bf7436123843d) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:40.113822 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:40.115285 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab360a3-c1d3-4dac-86bc-903620b8aadf None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:40.121742 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 622/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:24:39 2022] GET /identity/v3/role_assignments?user.id=a97c1ece267e4ca8b01bf7436123843d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=0f35caa7049f454eb6e09cd12fe8a8c2 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:42.598329 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43392f36-c820-4f73-8ac6-7f5bea5e1c16 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.598925 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43392f36-c820-4f73-8ac6-7f5bea5e1c16 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.598925 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43392f36-c820-4f73-8ac6-7f5bea5e1c16 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.600326 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 625/1247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:42.606660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.607179 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.607179 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.612394 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.625829 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:42.627044 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:42.700355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 623/1248] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:24:42.723726 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43392f36-c820-4f73-8ac6-7f5bea5e1c16 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:42.743066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:42.833044 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43392f36-c820-4f73-8ac6-7f5bea5e1c16 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:42.871276 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:42.872077 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:42.872362 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:42.872655 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:42.883492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:42.891344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:42.892488 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:42.893548 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 626/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:42.898567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.005461 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830bb6a-6b45-4346-bc00-63c61400b57a None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.040441 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.045452 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.046132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.046437 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.046711 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.047462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.048005 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.049601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.060559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 624/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.066720 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.152536 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.184200 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5694-85da-457a-b676-11b5c7fb0010 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.221041 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.222179 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.222628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.223086 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.236902 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.238495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:43.259109 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:43.260261 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:43.264081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:43.264081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:43.264081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:43.264081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:43.264081 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 627/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:43.267050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.383762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf8568-d1bc-496b-b0e9-5c89307cc56d None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.420565 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.421166 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.421493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.421733 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.422515 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.422829 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.424244 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.436029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 625/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.442143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.555348 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72fc05-743c-468d-a1a2-8ff789a81e11 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.582026 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.590974 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.591863 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.592139 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.592406 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.600783 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.602937 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:43.618291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:43.619503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:43.620688 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 628/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:24:43.625785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.700372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:43.752225 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab04587-1b56-4835-a861-ad9c2a753e17 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.791058 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.791743 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.792021 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.792339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:43.793304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:43.793622 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:43.795092 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:43.812306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 626/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:43.818435 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:43.932491 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f034e-c954-4a0e-ba39-0581611bd117 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:43.969311 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:43.969854 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:43.970134 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:43.970437 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.028568 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.043820 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.045347 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.151190 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 629/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:24:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:24:44.158045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.273915 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f25aa-a987-4084-bd8a-0fd3f022f5cf None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.288700 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.312959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.313912 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.314194 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.314469 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.326002 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.327498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:44.346908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:44.348027 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:44.349126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:44.349126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:44.349126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:44.349126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:44.349126 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 627/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:44.355818 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.449945 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.463770 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7295054-5c59-418f-88ac-1832668e4150 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.501650 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.502330 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.502562 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.502857 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.503576 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:44.503878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.505264 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.517847 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 630/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:44.524009 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.628845 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69738e7-7f02-4baa-996c-64b8bea179d9 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.664110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.664920 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.665374 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.665552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.673026 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.674594 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:44.689561 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:44.690618 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:44.691748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 628/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:44.698798 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.807103 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a343e11-1d86-4761-96d6-d9fc97f252c0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:44.841232 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:44.842008 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:44.842280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:44.842550 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:44.843354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:44.843658 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:44.845143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779995a-bd18-45a7-b9bb-4d284a7142fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:44.857120 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 631/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:44.863762 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:44.891793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:44.977624 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01b101-0285-405e-a500-3bc41413cd38 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:45.012724 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:45.013403 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:45.013686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:45.013961 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:45.014793 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=a97c1ece267e4ca8b01bf7436123843d) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:45.015112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:45.016580 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e658d81-3f75-402d-9af1-c1b147fb52f4 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:45.022350 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 629/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:24:44 2022] GET /identity/v3/role_assignments?user.id=a97c1ece267e4ca8b01bf7436123843d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:47.182170 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7472d-0649-4531-b282-e95331d31c4c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.182976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7472d-0649-4531-b282-e95331d31c4c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.182976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7472d-0649-4531-b282-e95331d31c4c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.185212 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 632/1261] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:47.192261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.192667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.192760 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.212128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:47.213398 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:47.285128 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 630/1262] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:47.308267 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7472d-0649-4531-b282-e95331d31c4c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.322300 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.428942 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7472d-0649-4531-b282-e95331d31c4c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.464421 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.465148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.465462 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.465731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.478579 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:47.486172 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:47.487160 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:47.488875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 633/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:47.493478 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.512780 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.595392 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb55c88-bb45-405e-891e-4439397e2b5c None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.640061 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.641070 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.641514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.641788 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] PATH_INFO: `/v3/roles` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.642538 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:47.642840 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.644242 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.653060 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 631/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:47.659564 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.740679 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.775258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec9f6-722d-428c-a2ca-1b8e977c4645 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.810951 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.811745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.812023 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.812297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:47.824582 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:47.827624 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:47.846673 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:47.847800 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:47.849954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:47.849954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:47.849954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:47.849954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:47.849954 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 634/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:47.856421 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:47.942095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:47.962236 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d706-81d5-4b39-bf36-bcee4e217272 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:47.998549 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:47.999147 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:47.999424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:47.999698 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.000424 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.000785 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.002212 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.013954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 632/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.020006 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.131821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f29e6-8e86-4446-b571-824534b9e999 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.151904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.169731 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.170538 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.170813 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.171087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.179737 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.181178 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:48.196424 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.198036 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:48.199322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 635/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.204116 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.312154 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852e0b5-eb69-4628-858e-8238bf73e3ad None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.348157 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.348805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.349079 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.349378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] PATH_INFO: `/v3/users` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.350206 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.350508 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.351990 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.360652 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.369356 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 633/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.375530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88a4d2-5dda-420a-9c53-df271578bf0b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.532470 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.533263 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.533571 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.533840 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.544694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.546177 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:48.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.564352 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.565546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:48.565546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:48.565546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:48.565546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:48.569445 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 636/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.571658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.678954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d9ace-c853-4957-a55a-9df7999b5da8 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.720401 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.721051 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.721353 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.721623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.722343 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:48.722642 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.724016 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.735115 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 634/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:48.741896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.781280 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:48.866281 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43193263-cd04-4df6-9ab8-b873df7ccbe7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:48.907297 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:48.908088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:48.908362 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:48.908628 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:48.915331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:48.916776 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:48.931246 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:48.932247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:48.933334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 637/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:48.939271 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:48.999627 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.048484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f40c0e-226e-4ae8-95ce-b55d4e5f6862 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.085332 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.085974 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.086252 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.086518 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.087293 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:49.087597 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.089053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b539625-49a9-4e0b-9d4a-7564aee4881e None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.101008 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 635/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:49.107733 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:49.209224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:49.219809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b120be-c5fa-4f48-a444-1ed89edac20d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:49.255814 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012be9-5f00-4869-ae15-c2f5f7e8eaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:49.256680 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012be9-5f00-4869-ae15-c2f5f7e8eaf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:49.256952 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012be9-5f00-4869-ae15-c2f5f7e8eaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:49.257218 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012be9-5f00-4869-ae15-c2f5f7e8eaf9 None admin] PATH_INFO: `/v3/projects/6aff3a02f4d0481b874e627706012719/users/a97c1ece267e4ca8b01bf7436123843d/roles/6426271bad544daf96fe8f3686d6eed5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:49.259260 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012be9-5f00-4869-ae15-c2f5f7e8eaf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aff3a02f4d0481b874e627706012719, user_id=a97c1ece267e4ca8b01bf7436123843d, role_id=6426271bad544daf96fe8f3686d6eed5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:49.260694 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012be9-5f00-4869-ae15-c2f5f7e8eaf9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:49.276678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 638/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:24:49 2022] PUT /identity/v3/projects/6aff3a02f4d0481b874e627706012719/users/a97c1ece267e4ca8b01bf7436123843d/roles/6426271bad544daf96fe8f3686d6eed5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:24:51.494658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c99c4-f8dd-442e-9ae7-2e28f5863311 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.495555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c99c4-f8dd-442e-9ae7-2e28f5863311 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.495555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c99c4-f8dd-442e-9ae7-2e28f5863311 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.497856 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 636/1274] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:51.504812 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.505678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.506471 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.529832 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:51.531035 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:51.615718 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 639/1275] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:51.650147 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c99c4-f8dd-442e-9ae7-2e28f5863311 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:51.779013 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c99c4-f8dd-442e-9ae7-2e28f5863311 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:51.796827 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:51.821689 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:51.822496 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.822778 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.823052 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:51.834297 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:24:51.841936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:51.843106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:51.844544 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 637/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:51.850002 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:51.945390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:51.965089 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98caa56-9f7b-45e8-99ca-5faa52ca6edc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:51.998578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:51.999230 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:51.999499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:51.999767 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.000585 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.000885 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.002283 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.010372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 640/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:24:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:24:52.018595 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.125748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39702b-ab1b-4027-87f4-b4cae4bda942 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.163042 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.163835 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.164113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.164393 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.175693 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.177165 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:52.195774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:52.196754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:52.196754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:52.196754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:52.196754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:52.196754 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 638/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:52.202618 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.307845 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbcd03-d619-4f9a-ae10-c12cb51c5d8b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.342546 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.343217 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.343493 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.343767 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.344492 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.344801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.346264 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.358034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 641/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:52.366621 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.402981 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.471028 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c613e-640f-413a-a82b-5a97039c63d3 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.507125 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.507959 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.508237 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.508509 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.516364 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.517854 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:24:52.540049 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:52.541197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:52.542304 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 639/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:52.547564 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.553522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fba674-f096-405f-bffd-d794c97c2196 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.699684 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.700358 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.700639 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.700908 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.701789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:52.702101 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.703542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:52.721337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 642/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:52.730257 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:52.830669 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:52.840783 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e90055-7bba-4cd2-b925-9069b37cff26 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:52.875904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:52.876551 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:52.876834 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:52.877108 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:52.947539 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:52.949105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.039969 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.053969 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 640/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:24:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:24:53.060054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.127075 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.169005 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa89fa-3fb4-4f3c-b994-452da3734c94 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.216442 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.217572 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.217936 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.218304 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.229046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.230532 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:24:53.248254 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:53.249359 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:53.250390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:24:53.250390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:24:53.250390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:24:53.250390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:53.250390 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 643/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:53.255414 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.362456 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144ed8-0309-43a2-879b-19d4b03cf793 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.397821 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.398472 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.398749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.399020 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] PATH_INFO: `/v3/domains` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.399712 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:53.400015 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.401428 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.412774 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 641/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:53.418565 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.526873 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6d93d-6845-43e2-b54e-b53ad78d51cd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.541213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.564956 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.565841 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.566110 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.566384 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.573998 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.575530 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:24:53.589668 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:53.590724 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application May 19 14:24:53.591663 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 644/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:53.596936 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.655297 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:53.707186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e24c1-ddda-424d-930d-f864e874e926 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.741955 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.742592 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.742892 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.743162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.743950 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:53.744262 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.745748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09cde1-57bf-4aec-8cba-584f7a9500e7 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.757819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 642/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:53.767570 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:53.874207 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac7b7-d08a-4f50-8ace-a52823a7f594 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:53.914959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:53.915604 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:53.915883 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:53.916155 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:53.916989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] RBAC: Adding query filter params (role.id=6426271bad544daf96fe8f3686d6eed5, scope.project.id=6aff3a02f4d0481b874e627706012719, user.id=a97c1ece267e4ca8b01bf7436123843d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:53.917335 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:53.918788 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f62711-738a-46ec-8879-a567ca6bd8c4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:53.925074 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 645/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:24:53 2022] GET /identity/v3/role_assignments?user.id=a97c1ece267e4ca8b01bf7436123843d&scope.project.id=6aff3a02f4d0481b874e627706012719&role.id=6426271bad544daf96fe8f3686d6eed5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:56.125967 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ba4ca-d2c5-45a8-83ca-349a613a869b None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.126563 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ba4ca-d2c5-45a8-83ca-349a613a869b None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.126911 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ba4ca-d2c5-45a8-83ca-349a613a869b None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.128251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 643/1288] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:56.134678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.135399 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.135542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.157664 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:56.158860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:56.191531 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.237143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 646/1289] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:56.259462 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ba4ca-d2c5-45a8-83ca-349a613a869b None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.379656 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ba4ca-d2c5-45a8-83ca-349a613a869b None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.391320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:56.422521 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.423324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.423601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.423871 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.424523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.425960 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:24:56.438188 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:24:56.439347 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:24:56.440370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:24:56.444397 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 644/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:24:56.445522 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.548860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98c7c-4233-4f23-837e-011b1dc4b795 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.590039 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.590701 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.590977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.591224 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.592004 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:56.592301 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.593702 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d8bc8-798e-4a35-8ec2-ebbb4ebc4053 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.598878 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 647/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:56.604274 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:56.734830 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ddd01-ec4d-4a12-87ee-ba2fa8deb143 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:56.772658 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:56.773351 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:56.773633 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:56.773903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] PATH_INFO: `/v3/services` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:56.774603 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:24:56.774904 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:56.776324 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65802ce-bff0-4ec6-974c-35c75a27261b None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:56.781296 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 645/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:24:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:24:59.017941 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e588e8-a733-4cb6-a010-ee7c97638ad0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.018508 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e588e8-a733-4cb6-a010-ee7c97638ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.018723 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e588e8-a733-4cb6-a010-ee7c97638ad0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.020990 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 648/1293] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:24:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:24:59.027555 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4f87-b8ac-4200-a49c-ba230f149668 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.028612 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4f87-b8ac-4200-a49c-ba230f149668 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.028876 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4f87-b8ac-4200-a49c-ba230f149668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.053205 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-5b2c4f87-b8ac-4200-a49c-ba230f149668 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:24:59.054461 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-5b2c4f87-b8ac-4200-a49c-ba230f149668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:24:59.085173 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:59.125110 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 646/1294] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:24:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:24:59.148615 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e588e8-a733-4cb6-a010-ee7c97638ad0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:24:59.181157 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:24:59.255318 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e588e8-a733-4cb6-a010-ee7c97638ad0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:24:59.291145 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64395a-1c43-4e09-bd4c-e9df0ff87fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:24:59.291809 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64395a-1c43-4e09-bd4c-e9df0ff87fbb None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:24:59.292087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64395a-1c43-4e09-bd4c-e9df0ff87fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:24:59.292354 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64395a-1c43-4e09-bd4c-e9df0ff87fbb None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:24:59.293074 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64395a-1c43-4e09-bd4c-e9df0ff87fbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:24:59.294484 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64395a-1c43-4e09-bd4c-e9df0ff87fbb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:24:59.309768 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 649/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:24:59 2022] POST /identity/v3/services => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:01.561977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c521-5358-4969-8e75-f5ce822a96d3 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.562581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c521-5358-4969-8e75-f5ce822a96d3 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.562581 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c521-5358-4969-8e75-f5ce822a96d3 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.564105 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 647/1296] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:01.570084 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.570559 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.570652 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.588296 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:01.589554 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:01.668033 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 650/1297] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:25:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:01.701580 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c521-5358-4969-8e75-f5ce822a96d3 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.816903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c521-5358-4969-8e75-f5ce822a96d3 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:01.854192 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:01.854998 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:01.855275 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:01.855533 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:01.856197 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:01.857639 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:01.869730 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:01.870882 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:25:01.871903 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 648/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:01.877465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:01.897258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:01.982944 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5055a8a-418a-473b-bd69-cdbca8ff9f7d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.017105 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.017797 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.018096 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.018314 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.019061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:02.019365 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.020745 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.026290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 651/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:25:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:02.033053 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.066476 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:02.149567 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542f268-29c0-4d66-9156-1e7b53e87ecb None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.183498 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.184163 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.184445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.184749 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.185577 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] RBAC: Adding query filter params (interface=public, service_id=97897b74aa6a4dec8a4c41c4b3c1fc44, region_id=RegionOne) {{(pid=80299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:02.185891 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.187286 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598efcf1-76a6-4969-a89b-24ed600d9279 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.192420 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 649/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/endpoints?service_id=97897b74aa6a4dec8a4c41c4b3c1fc44&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:02.198234 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:02.284000 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:02.303067 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf290b-cc6b-41eb-bfe1-9ac4c2bed653 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:02.337104 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae5024-b469-43fc-9f28-b45a31325809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:02.337819 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae5024-b469-43fc-9f28-b45a31325809 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:02.338100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae5024-b469-43fc-9f28-b45a31325809 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:02.338371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae5024-b469-43fc-9f28-b45a31325809 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:02.339064 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae5024-b469-43fc-9f28-b45a31325809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:02.340472 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae5024-b469-43fc-9f28-b45a31325809 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:02.347511 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 652/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:25:02 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:25:04.549201 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471b7b-2cbb-4e96-a051-b2795caa2f03 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.549898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471b7b-2cbb-4e96-a051-b2795caa2f03 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.549898 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471b7b-2cbb-4e96-a051-b2795caa2f03 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.551838 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 650/1302] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:25:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:25:04.560762 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.561184 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.561398 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.579775 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:25:04.580989 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:25:04.652356 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 653/1303] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:25:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:25:04.678635 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471b7b-2cbb-4e96-a051-b2795caa2f03 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:04.791564 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:04.799992 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471b7b-2cbb-4e96-a051-b2795caa2f03 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:04.834153 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:04.834970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.835251 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.835524 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.836182 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:04.837599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:25:04.849601 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:25:04.850831 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application with Lock( May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self._enter() May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application return self.creator() May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application created_value = creator( May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: ERROR keystone.server.flask.application May 19 14:25:04.852249 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 651/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:25:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:25:04.857143 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:04.936875 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:04.962159 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf434376-bac0-4af9-9544-899132cfc2ed None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:04.995592 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:04.996271 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:04.996551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:04.996821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:04.997563 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:25:04.997874 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:04.999259 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5a379-44ff-4c8b-8350-ca3b987fb194 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.004643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 654/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:25:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:25:05.010320 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:25:05.112505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72697a44-9f88-4a5d-86dd-13e8ce73d7a1 None admin] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:25:05.149893 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce32f84-fd53-4afa-b594-c304738c1582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '575363a61b9b436f9c41ee0b0af8ab40', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:25:05.150536 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce32f84-fd53-4afa-b594-c304738c1582 None admin] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:25:05.150809 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce32f84-fd53-4afa-b594-c304738c1582 None admin] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:25:05.151093 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce32f84-fd53-4afa-b594-c304738c1582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:25:05.151810 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce32f84-fd53-4afa-b594-c304738c1582 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:25:05.153512 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce32f84-fd53-4afa-b594-c304738c1582 None admin] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:25:05.173254 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 652/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:25:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:25:15.001766 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=30 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:25:15.168940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:09.136637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed10bb5-7175-4613-919d-37e1e7e33a58 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:09.137087 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed10bb5-7175-4613-919d-37e1e7e33a58 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:09.137171 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed10bb5-7175-4613-919d-37e1e7e33a58 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.144174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439b9a-33ad-4f91-8d44-51e8db7d5ba7 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:09.144608 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439b9a-33ad-4f91-8d44-51e8db7d5ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:09.144727 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439b9a-33ad-4f91-8d44-51e8db7d5ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.147534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 655/1307] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:09.238422 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-39439b9a-33ad-4f91-8d44-51e8db7d5ba7 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:09.239881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-39439b9a-33ad-4f91-8d44-51e8db7d5ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:09.355095 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 653/1308] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:26:09.405322 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb6a9-20f9-427d-8a94-19b96e98a33d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:09.405613 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb6a9-20f9-427d-8a94-19b96e98a33d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:09.406095 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb6a9-20f9-427d-8a94-19b96e98a33d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.407853 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 656/1309] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 14:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:09.414137 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc026b-0b70-413d-919c-8dacc637117d None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:09.414664 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc026b-0b70-413d-919c-8dacc637117d None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:09.414940 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc026b-0b70-413d-919c-8dacc637117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.495483 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-97dc026b-0b70-413d-919c-8dacc637117d None None] MFA Rules not processed for user `d8507240870340478b50c27ae54381ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:09.496619 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-97dc026b-0b70-413d-919c-8dacc637117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:09.526157 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:09.585570 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 654/1310] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 14:26:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:09.594302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb6a9-20f9-427d-8a94-19b96e98a33d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:09.596331 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6f7bb6a9-20f9-427d-8a94-19b96e98a33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:09.719693 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb6a9-20f9-427d-8a94-19b96e98a33d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:09.800643 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:09.801216 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:09.801518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:09.801791 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:09.803477 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:09.862899 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:09.921924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:09.923456 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fa087-e348-4adc-bcc5-6e0523920739 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:09.944891 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 657/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:12.404890 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0932a31-ba69-4058-b86a-c1dca5dc94b9 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:12.405334 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0932a31-ba69-4058-b86a-c1dca5dc94b9 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:12.405505 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0932a31-ba69-4058-b86a-c1dca5dc94b9 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:12.407029 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 655/1312] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:12.412374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804dc7-c077-4918-92cf-ba69b3aa024f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:12.412771 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804dc7-c077-4918-92cf-ba69b3aa024f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:12.413113 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804dc7-c077-4918-92cf-ba69b3aa024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:12.514960 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b1804dc7-c077-4918-92cf-ba69b3aa024f None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:12.518344 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b1804dc7-c077-4918-92cf-ba69b3aa024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:12.626541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 658/1313] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:26:12.677122 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683b5a-7af2-418a-8313-4dcc0b8c323d None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:12.677482 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683b5a-7af2-418a-8313-4dcc0b8c323d None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:12.677748 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a683b5a-7af2-418a-8313-4dcc0b8c323d None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:12.679568 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 656/1314] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 14:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:12.686290 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e8c6-2981-4b76-bf89-0f23c235d153 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:12.687210 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e8c6-2981-4b76-bf89-0f23c235d153 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:12.687982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e8c6-2981-4b76-bf89-0f23c235d153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:12.701601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:12.774977 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6511e8c6-2981-4b76-bf89-0f23c235d153 None None] MFA Rules not processed for user `d8507240870340478b50c27ae54381ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:12.776094 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6511e8c6-2981-4b76-bf89-0f23c235d153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:12.857871 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 659/1315] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:12.866664 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683b5a-7af2-418a-8313-4dcc0b8c323d None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:12.868255 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-0a683b5a-7af2-418a-8313-4dcc0b8c323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:12.988394 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a683b5a-7af2-418a-8313-4dcc0b8c323d None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:13.068082 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:13.070957 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:13.071620 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:13.071900 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:13.072174 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:13.074317 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:13.192024 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:13.193582 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a72c2-95f6-4781-87b5-b7304ecb19d3 service neutron] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:13.214411 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 657/1316] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:15.974558 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1482bb0-e729-4820-8d46-c020964ce21e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:15.975053 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1482bb0-e729-4820-8d46-c020964ce21e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:15.975053 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1482bb0-e729-4820-8d46-c020964ce21e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:15.976495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 660/1317] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:15.982092 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f546f4d-41c8-4a79-a9da-2a9da31138c9 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:15.982533 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f546f4d-41c8-4a79-a9da-2a9da31138c9 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:15.982631 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f546f4d-41c8-4a79-a9da-2a9da31138c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:16.053455 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-9f546f4d-41c8-4a79-a9da-2a9da31138c9 None None] MFA Rules not processed for user `dbc93960a58a448881c8655e47aa3062`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:16.054653 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-9f546f4d-41c8-4a79-a9da-2a9da31138c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:16.106045 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:16.158702 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 658/1318] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:16.201705 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1482bb0-e729-4820-8d46-c020964ce21e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:16.204191 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f1482bb0-e729-4820-8d46-c020964ce21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:16.316020 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1482bb0-e729-4820-8d46-c020964ce21e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:16.318932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:16.392199 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:16.392836 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:16.393111 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:16.393407 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:16.395046 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:16.515380 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:16.516821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e116af-7d2b-4a9e-b592-bfe42bfc6985 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:16.536417 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 661/1319] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:19.356427 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef605-86b0-4d30-8198-86522bc50e85 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.357306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef605-86b0-4d30-8198-86522bc50e85 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.357306 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef605-86b0-4d30-8198-86522bc50e85 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.359525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 659/1320] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:19.367815 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707e3e9-eae4-473f-aaf7-cbc38a0090dc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.368284 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707e3e9-eae4-473f-aaf7-cbc38a0090dc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.368378 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707e3e9-eae4-473f-aaf7-cbc38a0090dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.408897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:19.435676 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a707e3e9-eae4-473f-aaf7-cbc38a0090dc None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:19.436959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a707e3e9-eae4-473f-aaf7-cbc38a0090dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:19.540292 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 662/1321] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:26:19.582261 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acef605-86b0-4d30-8198-86522bc50e85 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:19.584623 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-8acef605-86b0-4d30-8198-86522bc50e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:19.702572 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acef605-86b0-4d30-8198-86522bc50e85 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:19.778289 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:19.778925 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:19.779200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:19.779470 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:19.781112 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:19.788682 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:19.894262 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:19.895716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060063f4-4510-4399-874f-3f3fbad48ee4 service neutron] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:19.915819 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 660/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:22.817586 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a0f51-f0b2-484a-b0a7-888bcd1bc2f7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:22.818154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a0f51-f0b2-484a-b0a7-888bcd1bc2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:22.818154 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a0f51-f0b2-484a-b0a7-888bcd1bc2f7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:22.819503 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 663/1323] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:22.825063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa37621-68f5-4d16-a8fe-0b23e07d0fdb None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:22.825517 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa37621-68f5-4d16-a8fe-0b23e07d0fdb None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:22.825629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa37621-68f5-4d16-a8fe-0b23e07d0fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:22.894231 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-1aa37621-68f5-4d16-a8fe-0b23e07d0fdb None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:22.895429 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-1aa37621-68f5-4d16-a8fe-0b23e07d0fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:22.959787 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:23.000245 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 661/1324] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:23.036622 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a0f51-f0b2-484a-b0a7-888bcd1bc2f7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:23.038213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-984a0f51-f0b2-484a-b0a7-888bcd1bc2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:23.140959 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a0f51-f0b2-484a-b0a7-888bcd1bc2f7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:23.153992 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:23.217465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:23.218119 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:23.218371 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:23.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:23.220353 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:23.330588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:23.332040 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5230d0-5301-4737-a619-8bd649875517 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:23.352601 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 664/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:25.611162 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46b705-16ca-4198-8638-16be361a1713 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:25.611770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46b705-16ca-4198-8638-16be361a1713 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:25.611770 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46b705-16ca-4198-8638-16be361a1713 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:25.613090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 662/1326] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:25.618495 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3cc67-0808-4766-b42e-473f210c4e25 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:25.618924 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3cc67-0808-4766-b42e-473f210c4e25 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:25.619034 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3cc67-0808-4766-b42e-473f210c4e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:25.668247 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:25.689976 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-37b3cc67-0808-4766-b42e-473f210c4e25 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:25.691185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-37b3cc67-0808-4766-b42e-473f210c4e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:25.790361 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 665/1327] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:25.826063 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46b705-16ca-4198-8638-16be361a1713 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:25.827593 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-0c46b705-16ca-4198-8638-16be361a1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:25.941260 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46b705-16ca-4198-8638-16be361a1713 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:26.015493 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:26.016146 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:26.016426 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:26.016722 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:26.018408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:26.035858 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:26.130908 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:26.132360 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3efc49-0f0e-4404-b322-7510753443bc service neutron] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:26.152525 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 663/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:28.433789 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c239b-c270-414c-8b10-a55aec920b85 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:28.434356 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c239b-c270-414c-8b10-a55aec920b85 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:28.434356 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c239b-c270-414c-8b10-a55aec920b85 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:28.435793 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 666/1329] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:28.441644 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156bf67-8cde-462c-9ae1-8355ba252812 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:28.442109 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156bf67-8cde-462c-9ae1-8355ba252812 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:28.442239 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156bf67-8cde-462c-9ae1-8355ba252812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:28.513338 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-b156bf67-8cde-462c-9ae1-8355ba252812 None None] MFA Rules not processed for user `dbc93960a58a448881c8655e47aa3062`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:28.514559 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-b156bf67-8cde-462c-9ae1-8355ba252812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:28.589744 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:28.629551 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 664/1330] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:28.671499 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c239b-c270-414c-8b10-a55aec920b85 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:28.673136 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-546c239b-c270-414c-8b10-a55aec920b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:28.794598 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c239b-c270-414c-8b10-a55aec920b85 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:28.815245 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:28.875877 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:28.876542 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:28.876817 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:28.877083 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:28.878787 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:28.996372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:28.998045 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c41a77-8554-4e2a-ad43-b3df062a9959 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:29.018360 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 667/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:31.472090 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4692c-27d5-4d20-9967-9e5369035558 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:31.472616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4692c-27d5-4d20-9967-9e5369035558 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:31.472616 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4692c-27d5-4d20-9967-9e5369035558 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:31.473962 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 665/1332] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:31.480910 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f997f19-8b07-4b90-a06a-2e640ad2f1e2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:31.481645 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f997f19-8b07-4b90-a06a-2e640ad2f1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:31.481810 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f997f19-8b07-4b90-a06a-2e640ad2f1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:31.552066 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:31.556465 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4f997f19-8b07-4b90-a06a-2e640ad2f1e2 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:31.558637 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4f997f19-8b07-4b90-a06a-2e640ad2f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:31.664699 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 668/1333] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:31.701485 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4692c-27d5-4d20-9967-9e5369035558 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:31.703084 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-4cd4692c-27d5-4d20-9967-9e5369035558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:31.810120 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4692c-27d5-4d20-9967-9e5369035558 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:31.889731 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:31.890418 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:31.890695 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:31.890970 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:31.892660 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:31.923690 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:32.012839 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:32.014335 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eaff4-8188-457d-9fa4-c29c84d1e32f service neutron] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:32.035132 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 666/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:34.269578 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ab780-2ff6-4c3c-be6f-0c5f8347ff69 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:34.270429 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ab780-2ff6-4c3c-be6f-0c5f8347ff69 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:34.270429 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ab780-2ff6-4c3c-be6f-0c5f8347ff69 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:34.272485 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 669/1335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:34.281634 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb607-8928-4422-81f3-b1c414384465 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:34.282379 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb607-8928-4422-81f3-b1c414384465 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:34.282471 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb607-8928-4422-81f3-b1c414384465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:34.357270 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-02dcb607-8928-4422-81f3-b1c414384465 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:34.358676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-02dcb607-8928-4422-81f3-b1c414384465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:34.439626 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:34.468189 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 667/1336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:26:34.514150 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ab780-2ff6-4c3c-be6f-0c5f8347ff69 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:34.516687 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab0ab780-2ff6-4c3c-be6f-0c5f8347ff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:34.640967 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ab780-2ff6-4c3c-be6f-0c5f8347ff69 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:34.672681 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:34.741152 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:34.742076 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:34.742334 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:34.742644 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:34.745760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:34.863001 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:34.864434 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0527112-6276-4b70-a9c0-2dbbbe157472 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:34.883600 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 670/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:37.757491 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f36c-117c-43c9-9b5a-c539e8947cf7 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:37.758435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f36c-117c-43c9-9b5a-c539e8947cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:37.758435 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f36c-117c-43c9-9b5a-c539e8947cf7 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:37.760599 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 668/1338] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:37.769612 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59d7b5-9eac-47c3-881b-bd72a5d1034a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:37.770061 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59d7b5-9eac-47c3-881b-bd72a5d1034a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:37.770166 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59d7b5-9eac-47c3-881b-bd72a5d1034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:37.843372 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3c59d7b5-9eac-47c3-881b-bd72a5d1034a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:37.844722 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3c59d7b5-9eac-47c3-881b-bd72a5d1034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:37.853588 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:37.955125 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 671/1339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:37.993790 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4f36c-117c-43c9-9b5a-c539e8947cf7 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:37.995372 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-6bd4f36c-117c-43c9-9b5a-c539e8947cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:38.109003 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4f36c-117c-43c9-9b5a-c539e8947cf7 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:38.194855 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:38.195499 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:38.195771 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:38.196034 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:38.197716 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:38.233646 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:38.310408 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:38.311838 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad225f4-0e93-45a0-98b3-70bfb20b4e3a service neutron] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:38.329598 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 669/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:42.724180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d449dd-c1f5-44be-8920-fb4d4e695097 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:42.724801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d449dd-c1f5-44be-8920-fb4d4e695097 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:42.724801 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d449dd-c1f5-44be-8920-fb4d4e695097 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:42.726144 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 672/1341] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:42.731667 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30607fd-00ae-4839-bd7c-36cf9e7c0764 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:42.732106 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30607fd-00ae-4839-bd7c-36cf9e7c0764 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:42.732200 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30607fd-00ae-4839-bd7c-36cf9e7c0764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:42.803717 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-f30607fd-00ae-4839-bd7c-36cf9e7c0764 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:42.804824 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-f30607fd-00ae-4839-bd7c-36cf9e7c0764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:42.902972 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=100 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:42.918310 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 670/1342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:42.960760 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d449dd-c1f5-44be-8920-fb4d4e695097 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:42.963374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c7d449dd-c1f5-44be-8920-fb4d4e695097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:43.064509 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d449dd-c1f5-44be-8920-fb4d4e695097 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:43.068313 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6026c-8d11-44b8-b186-f556dd3e022d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:43.068997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6026c-8d11-44b8-b186-f556dd3e022d service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:43.069285 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6026c-8d11-44b8-b186-f556dd3e022d service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:43.069564 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6026c-8d11-44b8-b186-f556dd3e022d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:43.079054 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6026c-8d11-44b8-b186-f556dd3e022d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:43.080534 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6026c-8d11-44b8-b186-f556dd3e022d service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:43.090268 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 673/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:45.867425 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d4c0d-4fa0-4f78-b339-63cc51b3f84c None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:45.867977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d4c0d-4fa0-4f78-b339-63cc51b3f84c None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:45.867977 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d4c0d-4fa0-4f78-b339-63cc51b3f84c None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:45.869322 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 671/1344] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:45.874678 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935c043-77f0-4e83-b872-8f6b9ca897ba None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:45.875122 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935c043-77f0-4e83-b872-8f6b9ca897ba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:45.875312 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935c043-77f0-4e83-b872-8f6b9ca897ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:45.904920 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9935c043-77f0-4e83-b872-8f6b9ca897ba None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:45.906148 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9935c043-77f0-4e83-b872-8f6b9ca897ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:45.938665 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:26:45.991279 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 674/1345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:46.028734 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d4c0d-4fa0-4f78-b339-63cc51b3f84c None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:46.030721 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-ba1d4c0d-4fa0-4f78-b339-63cc51b3f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:46.046196 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d4c0d-4fa0-4f78-b339-63cc51b3f84c None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:46.050262 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536c682-acdc-4c6a-9eaf-44a76a461a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:46.050907 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536c682-acdc-4c6a-9eaf-44a76a461a50 service neutron] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:46.051186 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536c682-acdc-4c6a-9eaf-44a76a461a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:46.051459 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536c682-acdc-4c6a-9eaf-44a76a461a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:46.060810 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536c682-acdc-4c6a-9eaf-44a76a461a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:46.062300 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536c682-acdc-4c6a-9eaf-44a76a461a50 service neutron] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:46.072042 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 672/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:50.189932 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8f4cc-1d55-4013-8b56-f3399f24e8b5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:50.190476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8f4cc-1d55-4013-8b56-f3399f24e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:50.190476 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8f4cc-1d55-4013-8b56-f3399f24e8b5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:50.191856 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 675/1347] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:50.197500 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618adbf-0024-4aa3-a300-04e047675587 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:50.197946 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618adbf-0024-4aa3-a300-04e047675587 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:50.198050 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618adbf-0024-4aa3-a300-04e047675587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:50.216980 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-2618adbf-0024-4aa3-a300-04e047675587 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:50.218195 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-2618adbf-0024-4aa3-a300-04e047675587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:50.226552 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 673/1348] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:50.266466 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8f4cc-1d55-4013-8b56-f3399f24e8b5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:26:50.278929 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8f4cc-1d55-4013-8b56-f3399f24e8b5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:26:50.282888 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d531a7-12c4-479c-9eb8-169321065299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8507240870340478b50c27ae54381ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:26:50.283541 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d531a7-12c4-479c-9eb8-169321065299 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:50.283821 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d531a7-12c4-479c-9eb8-169321065299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:50.284092 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d531a7-12c4-479c-9eb8-169321065299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:50.294195 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d531a7-12c4-479c-9eb8-169321065299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:26:50.295683 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d531a7-12c4-479c-9eb8-169321065299 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:26:50.305963 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 676/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:26:55.700779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67442892-8e29-4f36-bdc6-ca8cb0c217a1 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:55.701773 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67442892-8e29-4f36-bdc6-ca8cb0c217a1 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:55.702250 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67442892-8e29-4f36-bdc6-ca8cb0c217a1 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:55.704307 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 674/1350] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:55.709080 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d8d36-d757-46db-abbf-7185133a0973 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:55.709575 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d8d36-d757-46db-abbf-7185133a0973 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:55.709816 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d8d36-d757-46db-abbf-7185133a0973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:55.796213 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4c7d8d36-d757-46db-abbf-7185133a0973 None None] MFA Rules not processed for user `a97c1ece267e4ca8b01bf7436123843d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:26:55.797716 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4c7d8d36-d757-46db-abbf-7185133a0973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:26:55.884939 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 677/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:26:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:26:59.989022 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0915-8de9-4a40-be44-143f39757aa7 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:59.989917 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0915-8de9-4a40-be44-143f39757aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:59.990339 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0915-8de9-4a40-be44-143f39757aa7 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:26:59.992140 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 675/1352] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:26:59.997374 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080e860-35ee-4c4b-b57f-74fca123f4ae None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:26:59.997846 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080e860-35ee-4c4b-b57f-74fca123f4ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:26:59.998142 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080e860-35ee-4c4b-b57f-74fca123f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:00.018258 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e080e860-35ee-4c4b-b57f-74fca123f4ae None None] MFA Rules not processed for user `a97c1ece267e4ca8b01bf7436123843d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:00.019641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e080e860-35ee-4c4b-b57f-74fca123f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:00.029997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 678/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:00.211676 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=13 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:01.640779 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9013c-a899-4f47-bacb-904980042770 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:01.641474 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9013c-a899-4f47-bacb-904980042770 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:01.641474 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9013c-a899-4f47-bacb-904980042770 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:01.642816 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 676/1354] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:01.648185 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20005c2e-1493-4870-bfed-eaa95568ed92 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:01.648634 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20005c2e-1493-4870-bfed-eaa95568ed92 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:01.648735 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20005c2e-1493-4870-bfed-eaa95568ed92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:01.668412 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-20005c2e-1493-4870-bfed-eaa95568ed92 None None] MFA Rules not processed for user `a97c1ece267e4ca8b01bf7436123843d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:01.670479 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-20005c2e-1493-4870-bfed-eaa95568ed92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:01.682518 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 679/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:27:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:05.563913 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b398f-3c23-4d76-8be6-b05683c254f4 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:05.564884 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b398f-3c23-4d76-8be6-b05683c254f4 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:05.565431 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b398f-3c23-4d76-8be6-b05683c254f4 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:05.570973 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 677/1356] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:27:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:05.572337 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203228-8520-4816-85f5-b07d5982a5c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:05.572734 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203228-8520-4816-85f5-b07d5982a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:05.572897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203228-8520-4816-85f5-b07d5982a5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:05.591338 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f5203228-8520-4816-85f5-b07d5982a5c8 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:05.592982 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f5203228-8520-4816-85f5-b07d5982a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:05.622128 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 680/1357] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:05.662823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7306f-05b2-4416-98ad-1d9025fb7954 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:05.663881 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7306f-05b2-4416-98ad-1d9025fb7954 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:05.664365 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7306f-05b2-4416-98ad-1d9025fb7954 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:05.666277 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 678/1358] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 14:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:05.681439 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d0e9-ac2d-45a0-a968-341f6eb0b28d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:05.682302 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d0e9-ac2d-45a0-a968-341f6eb0b28d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:05.682785 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d0e9-ac2d-45a0-a968-341f6eb0b28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:05.750934 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:05.762019 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4e90d0e9-ac2d-45a0-a968-341f6eb0b28d None None] MFA Rules not processed for user `b4d24bc13b5c469cade02763acf072bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:05.763373 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4e90d0e9-ac2d-45a0-a968-341f6eb0b28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:05.879836 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 681/1359] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 14:27:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:27:05.889571 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7306f-05b2-4416-98ad-1d9025fb7954 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:05.913484 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7306f-05b2-4416-98ad-1d9025fb7954 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:05.922370 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96538f9-f390-43c5-bfaa-3edf9f803664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d24bc13b5c469cade02763acf072bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:05.923396 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96538f9-f390-43c5-bfaa-3edf9f803664 service nova] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:05.923824 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96538f9-f390-43c5-bfaa-3edf9f803664 service nova] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:05.924258 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96538f9-f390-43c5-bfaa-3edf9f803664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:05.934954 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96538f9-f390-43c5-bfaa-3edf9f803664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:05.939381 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96538f9-f390-43c5-bfaa-3edf9f803664 service nova] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:05.950064 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 679/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:27:08.358860 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b0f2-8228-4183-91de-3493a08c8663 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:08.359380 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b0f2-8228-4183-91de-3493a08c8663 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:08.359380 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b0f2-8228-4183-91de-3493a08c8663 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:08.360867 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 682/1361] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:08.366680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e196-cb9b-4ca0-b184-b27c98365c95 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:08.366680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e196-cb9b-4ca0-b184-b27c98365c95 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:08.366680 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e196-cb9b-4ca0-b184-b27c98365c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:08.385823 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-4526e196-cb9b-4ca0-b184-b27c98365c95 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:08.387191 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-4526e196-cb9b-4ca0-b184-b27c98365c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:08.397282 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 680/1362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:08.441872 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b0f2-8228-4183-91de-3493a08c8663 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:08.457308 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b0f2-8228-4183-91de-3493a08c8663 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:08.462779 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0912-8622-46d1-86ab-4b35d327adf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d24bc13b5c469cade02763acf072bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:08.463641 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0912-8622-46d1-86ab-4b35d327adf5 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:08.463782 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0912-8622-46d1-86ab-4b35d327adf5 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:08.463997 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0912-8622-46d1-86ab-4b35d327adf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:08.473030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0912-8622-46d1-86ab-4b35d327adf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:08.474659 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0912-8622-46d1-86ab-4b35d327adf5 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:08.485100 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 683/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:27:10.982416 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e7404-13f6-4e78-b130-100f465a9ab3 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:10.982994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e7404-13f6-4e78-b130-100f465a9ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:10.982994 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e7404-13f6-4e78-b130-100f465a9ab3 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:10.984323 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 681/1364] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:10.989775 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be06cc-6260-4c9c-a02e-8ee374981db8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:10.990551 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be06cc-6260-4c9c-a02e-8ee374981db8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:10.990988 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be06cc-6260-4c9c-a02e-8ee374981db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:11.011149 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-12be06cc-6260-4c9c-a02e-8ee374981db8 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:11.012610 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12be06cc-6260-4c9c-a02e-8ee374981db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:11.021126 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 684/1365] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:11.067776 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c106e-e836-41ee-9f34-aaf75d9a7c42 None None] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:11.068107 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c106e-e836-41ee-9f34-aaf75d9a7c42 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:11.068332 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c106e-e836-41ee-9f34-aaf75d9a7c42 None None] PATH_INFO: `/` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:11.069697 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 682/1366] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 14:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:11.075453 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c634dc-898d-44eb-8b16-869303ef3080 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:11.075721 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c634dc-898d-44eb-8b16-869303ef3080 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:11.076135 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c634dc-898d-44eb-8b16-869303ef3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:11.095746 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-53c634dc-898d-44eb-8b16-869303ef3080 None None] MFA Rules not processed for user `b4d24bc13b5c469cade02763acf072bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:11.097030 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53c634dc-898d-44eb-8b16-869303ef3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:11.106829 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 685/1367] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:11.115006 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c106e-e836-41ee-9f34-aaf75d9a7c42 None None] Authenticating user token {{(pid=80299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:11.131368 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c106e-e836-41ee-9f34-aaf75d9a7c42 None None] Validating token access rules against request {{(pid=80299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:11.137492 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7de593-251b-4835-9b55-54f9c10289d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d24bc13b5c469cade02763acf072bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:11.138686 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de593-251b-4835-9b55-54f9c10289d4 service nova] REQUEST_METHOD: `GET` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:11.139152 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de593-251b-4835-9b55-54f9c10289d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:11.139629 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de593-251b-4835-9b55-54f9c10289d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:11.150217 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de593-251b-4835-9b55-54f9c10289d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:11.152805 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de593-251b-4835-9b55-54f9c10289d4 service nova] RBAC: Authorization granted {{(pid=80299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:11.164758 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 683/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:27:11.858897 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67cb8-3f51-4a18-89f7-2012d4202746 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:11.859180 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67cb8-3f51-4a18-89f7-2012d4202746 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:11.859498 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67cb8-3f51-4a18-89f7-2012d4202746 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:11.861088 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 686/1369] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:11.866378 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f609c-0328-425c-8fdb-070543603a60 None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:11.866775 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f609c-0328-425c-8fdb-070543603a60 None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:11.867222 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f609c-0328-425c-8fdb-070543603a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:11.887797 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-346f609c-0328-425c-8fdb-070543603a60 None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:11.889113 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-346f609c-0328-425c-8fdb-070543603a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:11.898514 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 684/1370] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:13.428474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5a685-afc3-431f-8c31-40db60272d3d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:13.429051 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5a685-afc3-431f-8c31-40db60272d3d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:13.429051 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5a685-afc3-431f-8c31-40db60272d3d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:13.430467 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 687/1371] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:27:13.435874 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f9340-f828-4671-8e22-4aa28f787b8a None None] REQUEST_METHOD: `POST` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:13.436342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f9340-f828-4671-8e22-4aa28f787b8a None None] SCRIPT_NAME: `/identity` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:13.436445 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f9340-f828-4671-8e22-4aa28f787b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:13.455355 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.auth.core [None req-d70f9340-f828-4671-8e22-4aa28f787b8a None None] MFA Rules not processed for user `575363a61b9b436f9c41ee0b0af8ab40`. Rule list: `[]` (Enabled: `True`). {{(pid=80299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:27:13.456523 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG keystone.common.fernet_utils [None req-d70f9340-f828-4671-8e22-4aa28f787b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:27:13.464342 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: [pid: 80299|app: 0|req: 685/1372] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:27:13.517085 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5a685-afc3-431f-8c31-40db60272d3d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:27:13.532879 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5a685-afc3-431f-8c31-40db60272d3d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:27:13.539003 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e84d2-b87f-4546-8c4f-84e54a04fa48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4d24bc13b5c469cade02763acf072bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aff3a02f4d0481b874e627706012719', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:27:13.539636 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e84d2-b87f-4546-8c4f-84e54a04fa48 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:27:13.539912 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e84d2-b87f-4546-8c4f-84e54a04fa48 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:27:13.540178 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e84d2-b87f-4546-8c4f-84e54a04fa48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:27:13.548686 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e84d2-b87f-4546-8c4f-84e54a04fa48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:27:13.550291 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e84d2-b87f-4546-8c4f-84e54a04fa48 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:27:13.559740 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 688/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:27:23.449221 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: DEBUG dbcounter [-] [80299] Writing DB stats keystone:SELECT=21 {{(pid=80299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:23.555904 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=54 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:27:35.739176 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: Thu May 19 14:27:35 2022 - graceful shutdown triggered... May 19 14:27:35.739353 dv-cinder-842542-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:27:35.740023 dv-cinder-842542-1-iscsi devstack@keystone.service[80299]: Gracefully killing worker 2 (pid: 80299)... May 19 14:27:35.740474 dv-cinder-842542-1-iscsi devstack@keystone.service[80298]: Gracefully killing worker 1 (pid: 80298)... May 19 14:27:36.741211 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: worker 1 buried after 1 seconds May 19 14:27:36.741211 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: worker 2 buried after 1 seconds May 19 14:27:36.741211 dv-cinder-842542-1-iscsi devstack@keystone.service[80288]: goodbye to uWSGI. May 19 14:27:36.742860 dv-cinder-842542-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:27:36.743404 dv-cinder-842542-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 14:33:27.940115 dv-cinder-842542-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:33:27.947714 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:33:27 2022] *** May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: nodename: dv-cinder-842542-1-iscsi May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: machine: x86_64 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: clock source: unix May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: pcre jit disabled May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: detected number of CPU cores: 4 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: current working directory: / May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: detected binary path: /usr/bin/uwsgi-core May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: your processes number limit is 47680 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: your memory page size is 4096 bytes May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: detected max file descriptor number: 2048 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: lock engine: pthread robust mutexes May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: thunder lock: enabled May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:33:27.952235 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:33:27.996003 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: Python main interpreter initialized at 0x55eb4f6929c0 May 19 14:33:27.996003 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: python threads support enabled May 19 14:33:27.996003 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: your server socket listen backlog is limited to 100 connections May 19 14:33:27.996003 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: your mercy for graceful operations on workers is 90 seconds May 19 14:33:27.996440 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:33:27.996440 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: *** Operational MODE: preforking *** May 19 14:33:27.996675 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: *** uWSGI is running in multiple interpreter mode *** May 19 14:33:27.996675 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: spawned uWSGI master process (pid: 150359) May 19 14:33:27.996900 dv-cinder-842542-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:33:27.998577 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: spawned uWSGI worker 1 (pid: 150371, cores: 1) May 19 14:33:27.998577 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: spawned uWSGI worker 2 (pid: 150372, cores: 1) May 19 14:33:27.998577 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:33:33.633618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:33:33.633618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: from cryptography.utils import int_from_bytes May 19 14:33:33.634538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:33:33.634538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: from cryptography.utils import int_from_bytes May 19 14:33:33.634691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:33:33.634691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: from cryptography.utils import int_from_bytes May 19 14:33:33.634691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:33:33.634691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: from cryptography.utils import int_from_bytes May 19 14:33:33.683357 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:33:33.683561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:33:33.683660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:33:33.683757 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] command line args: [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:33:33.683846 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:33:33.683938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] command line args: [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:33:33.684033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:33:33.684122 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ================================================================================ {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:33:33.684231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:33:33.684314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ================================================================================ {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:33:33.684403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] admin_token = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.684620 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] config_dir = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.684735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] admin_token = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.684818 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.684904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] config_dir = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.684984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] config_source = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] debug = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] config_source = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685687 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] debug = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685687 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.685949 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686071 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] insecure_debug = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] insecure_debug = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686594 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.686948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687195 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_config_append = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_config_append = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_dir = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687845 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_dir = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.687943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_file = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_file = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.688993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689341 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.689959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690047 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.690939 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] notification_format = cadf {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] notification_format = cadf {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] publish_errors = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.691931 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] publish_errors = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692527 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692614 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692813 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.692910 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] standard_threads = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693111 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] strict_password_check = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] standard_threads = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] strict_password_check = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] use_eventlog = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693706 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] use_journal = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.693917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] use_eventlog = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] use_json = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] use_journal = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] use_stderr = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] use_json = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] use_syslog = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] use_stderr = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] watch_log_file = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] use_syslog = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.694862 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.694947 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] watch_log_file = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:33:33.695035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695230 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695324 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695425 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695771 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.695959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696325 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696890 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.696987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697162 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697442 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.697914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698488 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.698980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699171 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699267 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699354 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.699941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700293 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700388 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700734 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.700936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701023 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701367 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701456 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701548 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.701933 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702179 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702392 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702493 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.702918 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703289 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703384 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703477 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.703930 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.704041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.704620 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.704725 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.704821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.704940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705152 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705462 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705574 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.705958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706052 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706689 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.706943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707110 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707303 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.707872 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708139 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708415 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.708953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.709979 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710113 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.enabled = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.710925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711135 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.enabled = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.711863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712601 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.712984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713671 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713871 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.713968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714487 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714575 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714741 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.714924 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715186 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715419 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.715996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716084 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716836 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.716946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717649 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.external = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.mapped = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.717973 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718171 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.external = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.password = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.mapped = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718818 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] auth.token = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.718905 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719133 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] catalog.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.password = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719631 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] auth.token = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.719923 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720050 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] catalog.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720475 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720671 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.720841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721185 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721329 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.721931 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.722994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723896 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.723999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724734 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.724940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725684 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725797 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.725938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726490 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.726977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.727899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728318 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728625 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728749 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.728857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729026 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729320 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729701 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729786 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.729910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730878 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.730993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731112 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731549 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731833 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.731940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.732934 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733167 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733319 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733643 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.733990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.734889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735254 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.password = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.735937 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736132 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736218 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736512 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.password = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736683 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.736941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737132 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737681 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.737926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738115 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738201 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.738939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739242 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739325 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739457 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.739919 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740181 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740658 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740882 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.740984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741872 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.741979 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742239 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.742945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.743955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744229 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744384 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744674 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.744934 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] policy.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745323 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745823 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] receipt.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.745909 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746060 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746160 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] policy.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746418 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746509 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746752 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.746851 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747160 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] receipt.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747790 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.747976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748387 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] revoke.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.748937 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749335 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] role.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] role.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] role.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749910 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] revoke.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.749998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] role.list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] role.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750698 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] role.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.750959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] role.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] role.list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751398 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.751895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.752908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753305 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753475 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.753949 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754286 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754640 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.754980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755119 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755690 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.755914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.756853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.757927 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758050 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758323 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758464 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.provider = fernet {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.758928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759262 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759495 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.provider = fernet {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.759975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] trust.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760431 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760795 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.760887 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] trust.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761519 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.761988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762236 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762330 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.connection = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762491 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762619 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.762941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763186 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.connection = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763644 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763797 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.763981 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764087 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764197 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764325 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764556 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.764900 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765021 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765115 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765627 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765871 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.765991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766456 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:33:33.766737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.766939 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.767171 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.767411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.767647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:33:33.767818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:33:33.799041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.799413 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.799511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.800495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.800723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.800943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.808332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.809697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.836423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.836660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.837496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.837724 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.888087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.888249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.938548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.938726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.938832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:33.938942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.001913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002112 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002319 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.002988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.003219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.025961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026541 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.026942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.027043 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.027120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.027206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.027300 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.027510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.038154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.038699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.052323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.052654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.052870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.052994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053385 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053488 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053742 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.053913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.054109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:33:34.070515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.070743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:33:34.070978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.071118 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.071198 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:33:34.071311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:33:34.071515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.071668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.071668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:33:34.071853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.071953 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.072030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.072205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.072347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.072441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.072701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.083179 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.083425 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.083695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.083909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.084078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.084235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.084398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.084556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.092195 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.092620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.095697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.095913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.096072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.096179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.096369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.096532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.096630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.096819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097346 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.097966 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.098122 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.098270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.098424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.098561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.098654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.098654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.124273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.124890 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.125890 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.126169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.126522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.126803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.131352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.131628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.131900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.132038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.132301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.132580 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.139133 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.139414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.139876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.140135 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.143956 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.144187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.144498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.144709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.144817 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145467 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.145904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.146099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.146099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.146398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.146588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.164619 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.164864 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.165621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.165899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.169468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.170524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.173062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.173379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.174177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.174475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.178337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.179458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.182030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.183140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.185768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.186033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.186331 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.186610 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.186904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.187114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.187286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.187389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.187680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.188000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.188357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.188642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.206092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.206451 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.206713 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.206991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.207490 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.207868 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.208142 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.208420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.215031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.216406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.216718 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.216994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.218165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.218442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.220056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.220328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.220614 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.220887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.221154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.221513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.221791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.222080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.222362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.222631 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.234380 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.234632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.234930 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.235205 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.235484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.235731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.236010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.236226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.236334 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.236540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.236823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.237105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.237376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.237679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.253939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.255722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.257168 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.259030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.260559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.262152 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.262316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.263791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.263947 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.264042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.264335 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.265604 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.265891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.266171 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.271641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.273202 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.273584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.275637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.276940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.277169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.277471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.277743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.278873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.279166 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.279451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.279732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.286834 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.287170 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.287488 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.288945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.289280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.289660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.295430 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=150371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.295714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.295990 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.296260 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.296529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.296811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.297074 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.297369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.297510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=150372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:33:34.297614 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.297822 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.297971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.298867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=150371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.299047 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.299138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.299410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.299697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.299969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.300255 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.300517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.300791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.301052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=150372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:33:34.337680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55eb4f6929c0 pid: 150371 (default app) May 19 14:33:34.339641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55eb4f6929c0 pid: 150372 (default app) May 19 14:33:34.341105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158123d-d0b5-44b0-89ae-05fa0f51cf5d None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:34.341421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158123d-d0b5-44b0-89ae-05fa0f51cf5d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:34.341697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158123d-d0b5-44b0-89ae-05fa0f51cf5d None None] PATH_INFO: `/v3` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:34.345935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Thu May 19 14:33:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:42.339767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:42.340396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:42.340396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:42.347664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:42.356405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:42.357144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:42.357366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:42.428890 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: INFO dbcounter [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] Registered counter for database keystone May 19 14:33:42.453850 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG oslo_db.sqlalchemy.engines [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=150371) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:33:42.464175 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writer thread running {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:33:42.666650 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG passlib.handlers.bcrypt [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=150371) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:33:42.667273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG passlib.handlers.bcrypt [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=150371) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:33:42.724597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:42.726315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:42.861926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:33:42.886271 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:42.959856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: INFO dbcounter [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] Registered counter for database keystone May 19 14:33:42.981767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG oslo_db.sqlalchemy.engines [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=150372) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:33:42.989988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writer thread running {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:33:43.120030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0af6e8-17a9-470b-80b1-4547a2a5e655 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:43.125720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7ef1b2ba8e64b46822150f5756f1d6e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:43.126411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:43.126698 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:43.126965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:43.136282 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:43.187562 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:33:43.187892 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe523e26-2ed8-4853-b3ec-42048b8fa680 admin admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:43.198548 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 14:33:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:33:43.205711 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:43.223751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2d1a-aae6-4c2a-bbd9-c2bc5628f812 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:43.228402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7ef1b2ba8e64b46822150f5756f1d6e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:43.229581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:43.229874 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:43.230162 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] PATH_INFO: `/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:43.231881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] RBAC: Authorizing `identity:get_project(project_id=e7ef1b2ba8e64b46822150f5756f1d6e)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:43.395232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: WARNING keystone.common.rbac_enforcer.enforcer [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:33:43.395973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab6c0-0632-46da-864c-d93edd581a1e admin admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:43.400853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Thu May 19 14:33:43 2022] GET /identity/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:45.579187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531aea-26b7-4ee3-977e-8cf1f5f6d8e9 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:45.580038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531aea-26b7-4ee3-977e-8cf1f5f6d8e9 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:45.580038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531aea-26b7-4ee3-977e-8cf1f5f6d8e9 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:45.587027 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:45.596395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:45.596828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:45.596994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:45.619462 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:45.621037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:45.630757 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:33:45.654264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531aea-26b7-4ee3-977e-8cf1f5f6d8e9 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:45.672486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531aea-26b7-4ee3-977e-8cf1f5f6d8e9 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:45.678841 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fec4f-c87b-4d3b-9330-962f264ffae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7ef1b2ba8e64b46822150f5756f1d6e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:45.679485 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fec4f-c87b-4d3b-9330-962f264ffae1 admin admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:45.679764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fec4f-c87b-4d3b-9330-962f264ffae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:45.680034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fec4f-c87b-4d3b-9330-962f264ffae1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:45.687991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fec4f-c87b-4d3b-9330-962f264ffae1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:45.689829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fec4f-c87b-4d3b-9330-962f264ffae1 admin admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:45.698983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 14:33:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:33:45.707756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:45.721455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a818e-18b9-4703-80ab-402ae3ad0696 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:45.727902 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5645a-cdd5-4fd9-8c34-6bf14dd4d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7ef1b2ba8e64b46822150f5756f1d6e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:45.728781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5645a-cdd5-4fd9-8c34-6bf14dd4d76e admin admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:45.729058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5645a-cdd5-4fd9-8c34-6bf14dd4d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:45.729423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5645a-cdd5-4fd9-8c34-6bf14dd4d76e admin admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:45.730979 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5645a-cdd5-4fd9-8c34-6bf14dd4d76e admin admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:45.732829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5645a-cdd5-4fd9-8c34-6bf14dd4d76e admin admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:45.734434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Thu May 19 14:33:45 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:48.010358 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942c8d-9634-4924-8aa1-390a2f961b7a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.010959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622a44f-500d-4b95-bbcb-e5cf6df8fbd9 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.011358 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942c8d-9634-4924-8aa1-390a2f961b7a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.011460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622a44f-500d-4b95-bbcb-e5cf6df8fbd9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.011566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942c8d-9634-4924-8aa1-390a2f961b7a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.011662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622a44f-500d-4b95-bbcb-e5cf6df8fbd9 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.012241 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:48.012401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:33:48.017406 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.017575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.017683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.017831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.017938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.018087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.035943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:48.037122 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:48.060826 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG passlib.handlers.bcrypt [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=150372) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:33:48.061270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG passlib.handlers.bcrypt [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=150372) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:33:48.107444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:48.109044 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:48.121993 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 7/12] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:48.144597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.169437 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:48.211569 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 6/13] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:48.234298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.248546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf33c83-a8ca-48b9-b8cd-92c93c435bec None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.285685 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.286277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.286555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.286824 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.356174 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:48.357756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:48.383318 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:48.449661 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35968f6-6efd-4a3e-8bbf-0f36273fec3e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.453831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:48.472351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 8/14] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:33:48.478837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.493560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.494635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.494919 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.495193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.507381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:33:48.519051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:33:48.520386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:33:48.521452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 7/15] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:33:48.526773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.581361 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e031d9e-a42f-452c-a8ff-4e3d02d1e267 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.603181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:48.615671 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.616483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.616768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.617132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.626869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc73f-64bd-419b-8219-4704933ebe51 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.627292 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:48.628793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:33:48.649508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:33:48.650801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:33:48.651754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:33:48.651754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:33:48.651754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:33:48.651754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:33:48.651754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 9/16] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:33:48.657228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.661411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.662098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.662386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.662669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.663442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:48.663754 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:48.665296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:48.677375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 8/17] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:48.683057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.709818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:48.764991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaab148-6e15-43f6-85f0-724ec3494298 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.788355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0823c-97af-47ca-8666-24e25daac105 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.804066 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.804765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.805051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.805363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.806096 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:48.806403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:48.807830 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:48.825554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 10/18] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:48.830575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.831532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.831755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.832027 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:48.833786 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:48.835291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:48.837523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 9/19] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:48.844272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:48.883228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:48.959260 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44664418-81c1-4269-92e5-fe31c4f94ba2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:48.997503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f521e2-c22c-4b33-86f3-e207899133da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:48.998316 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f521e2-c22c-4b33-86f3-e207899133da None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:48.998600 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f521e2-c22c-4b33-86f3-e207899133da None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:48.998853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f521e2-c22c-4b33-86f3-e207899133da None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:49.000103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f521e2-c22c-4b33-86f3-e207899133da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:49.001618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f521e2-c22c-4b33-86f3-e207899133da None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:49.003353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 11/20] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:49.009582 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:49.113799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:49.121948 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6928b35-0305-4171-80a7-0dfa482072eb None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:49.161143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:49.161874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:49.162161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:49.162439 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:49.163357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.domain.id=default, user.id=0a3ad0c94b4942efa0beedd41caa0934) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:49.163673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:49.165185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c93681-2ed6-456f-8657-74997c5ab438 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:49.175719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 10/21] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 14:33:49 2022] GET /identity/v3/role_assignments?user.id=0a3ad0c94b4942efa0beedd41caa0934&scope.domain.id=default&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:51.289995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd07a8-8857-43bb-8c35-85b90751ef56 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.291070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd07a8-8857-43bb-8c35-85b90751ef56 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.291568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd07a8-8857-43bb-8c35-85b90751ef56 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.293167 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 12/22] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:51.310954 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.311927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.312414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.338374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:51.339997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:51.425322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 11/23] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:51.459192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd07a8-8857-43bb-8c35-85b90751ef56 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:51.602270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd07a8-8857-43bb-8c35-85b90751ef56 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:51.604280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:51.650819 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:51.651646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.651957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.652202 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.666756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:51.668255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-90cde359-14a8-4b4c-961f-a318732bffe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:33:51.688343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:33:51.689402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:33:51.690371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:33:51.690371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:33:51.690371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:33:51.690371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:33:51.690371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 13/24] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:33:51.697311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:51.793798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca014c1d-cd3b-4b23-bf3a-7906206a4a6d None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.794135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca014c1d-cd3b-4b23-bf3a-7906206a4a6d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.794357 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca014c1d-cd3b-4b23-bf3a-7906206a4a6d None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.795745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 14/25] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:33:51.800973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105ebbd-ae06-4ca2-9a52-5e7e98c132ae None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.801333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105ebbd-ae06-4ca2-9a52-5e7e98c132ae None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.801556 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105ebbd-ae06-4ca2-9a52-5e7e98c132ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.830164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-5105ebbd-ae06-4ca2-9a52-5e7e98c132ae None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:51.831770 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-5105ebbd-ae06-4ca2-9a52-5e7e98c132ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:51.832311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:51.855553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8ecb8-bb3d-4e62-8462-f3d6444e34c1 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:51.918200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:51.919260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.919781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.920272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.921309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:51.921814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:51.923651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:51.966024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 12/26] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:51.972798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 15/27] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:33:51.974554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639048f7-50c8-445f-935e-842ee1984659 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.975623 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:51.976554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639048f7-50c8-445f-935e-842ee1984659 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.977438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639048f7-50c8-445f-935e-842ee1984659 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:51.980697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 16/28] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:51.987215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:51.987981 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:51.988472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.024936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:52.029448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:52.153479 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f30c-0eeb-4672-8bfc-b02dbd843280 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:52.163379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 17/29] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:33:52.164277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:52.189757 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:52.194683 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:52.195787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:52.196367 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:52.196969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.197941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:52.199733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:52.235666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:52.286013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccde7c-5bc6-40c2-9556-3481c4e300d9 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:52.321017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba99369-0a6c-466d-9931-66455465fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:52.321728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba99369-0a6c-466d-9931-66455465fa8d None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:52.322020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba99369-0a6c-466d-9931-66455465fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:52.322269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba99369-0a6c-466d-9931-66455465fa8d None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.323020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba99369-0a6c-466d-9931-66455465fa8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:52.324446 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba99369-0a6c-466d-9931-66455465fa8d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:52.364436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 13/30] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:33:51 2022] POST /identity/v3/projects => generated 301 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:33:52.370570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:52.459999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 18/31] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:33:52 2022] POST /identity/v3/roles => generated 219 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:33:52.459999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f60534-b105-45fa-8b37-3681913d947c None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:52.459999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f60534-b105-45fa-8b37-3681913d947c None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:52.459999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f60534-b105-45fa-8b37-3681913d947c None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.460445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 19/32] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:52.473264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:52.473264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:52.473264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.505815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:52.507900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:52.563102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8fe2-cee3-440e-bdb7-b0832df48a80 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:52.591907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 20/33] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:52.616467 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:52.617535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:52.617965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:52.618342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.619294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:52.621049 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:52.631569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:52.636345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:52.653356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 14/34] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:33:52 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:33:52.792613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ba42-5547-43ec-9438-d03b78d4d3de None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:52.832614 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b82999-33ed-4f5c-b23f-24cd20c4e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:52.833951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82999-33ed-4f5c-b23f-24cd20c4e0cd None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:52.834480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82999-33ed-4f5c-b23f-24cd20c4e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:52.834984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82999-33ed-4f5c-b23f-24cd20c4e0cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:52.849347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b82999-33ed-4f5c-b23f-24cd20c4e0cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-33b82999-33ed-4f5c-b23f-24cd20c4e0cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:33:52.859126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:33:52.860258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:33:52.861165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:33:52.864032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 21/35] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:33:52.870444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:52.964126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:53.017286 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae524fdf-68d8-426a-891b-8da6f5e2cd54 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.037705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d47294-de3b-400b-a7a9-1246743e8426 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.038381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d47294-de3b-400b-a7a9-1246743e8426 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.039610 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d47294-de3b-400b-a7a9-1246743e8426 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.041621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 22/36] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:53.047409 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.048393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.048680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.062500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.063652 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.064158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.064684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.065809 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:53.066306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.067983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.071301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:53.072607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:53.081903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 15/37] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:53.089154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:53.140853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:53.154447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 23/38] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:53.177014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:53.202018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752bd7d-ab65-40f5-9ec2-087a9b6c7add None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.227546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:53.243986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.245447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.245897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.246177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.247532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.249060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.250874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 16/39] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:33:53 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:53.256614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:53.297145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb029-2fea-4ff9-b091-f753363401bf None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.335397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.336079 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.336534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.336534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.337342 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.338927 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.360473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debece5-22ee-471c-a4e3-380a5bd0e243 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.362926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 24/40] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:33:53 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:33:53.400808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb826cd-6109-4080-818f-455a94d10898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.401662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb826cd-6109-4080-818f-455a94d10898 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.401966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb826cd-6109-4080-818f-455a94d10898 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.402200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb826cd-6109-4080-818f-455a94d10898 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.403601 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb826cd-6109-4080-818f-455a94d10898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.405063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb826cd-6109-4080-818f-455a94d10898 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.406774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 17/41] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:53.412675 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:53.419582 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b871b0-c760-4d02-96fd-6423674524c1 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.419859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b871b0-c760-4d02-96fd-6423674524c1 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.420148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b871b0-c760-4d02-96fd-6423674524c1 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.421884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 18/42] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:53.427857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.428203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.428432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.447166 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:53.448801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:53.480109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:53.513906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:53.530650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 19/43] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:53.545653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a3fd-38ab-4c03-9284-efedbd42d04a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.554044 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:53.588941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.589856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.590173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.590503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] PATH_INFO: `/v3/domains/default/users/0a3ad0c94b4942efa0beedd41caa0934/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.592641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0a3ad0c94b4942efa0beedd41caa0934, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.594157 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.610954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 25/44] 10.222.0.73 () {58 vars in 1361 bytes} [Thu May 19 14:33:53 2022] PUT /identity/v3/domains/default/users/0a3ad0c94b4942efa0beedd41caa0934/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:33:53.663914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fc87a-db1f-4b20-8231-3097f66eb531 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.704377 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c01d7-371c-43cd-af9a-abb82cb16924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.705214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c01d7-371c-43cd-af9a-abb82cb16924 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.705531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c01d7-371c-43cd-af9a-abb82cb16924 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.705808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c01d7-371c-43cd-af9a-abb82cb16924 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.707081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c01d7-371c-43cd-af9a-abb82cb16924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.708605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c01d7-371c-43cd-af9a-abb82cb16924 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.710401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 20/45] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:53.716301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:53.825478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948776fb-42ef-462d-8c11-5f5e5393d6a8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:53.839365 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:53.862409 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e6957-869c-4b96-ab6d-6e1d81696a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:53.863115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e6957-869c-4b96-ab6d-6e1d81696a40 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:53.863398 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e6957-869c-4b96-ab6d-6e1d81696a40 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:53.863669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e6957-869c-4b96-ab6d-6e1d81696a40 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:53.864386 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e6957-869c-4b96-ab6d-6e1d81696a40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:53.865937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e6957-869c-4b96-ab6d-6e1d81696a40 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:53.893951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 26/46] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:33:53 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:33:55.831065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cced58d-2373-47e3-91c9-fec48b8664f2 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:55.831622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cced58d-2373-47e3-91c9-fec48b8664f2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:55.831622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cced58d-2373-47e3-91c9-fec48b8664f2 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:55.832990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 21/47] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:55.838661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b61b4-c059-4be7-8564-28ded595508b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:55.839066 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b61b4-c059-4be7-8564-28ded595508b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:55.839224 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b61b4-c059-4be7-8564-28ded595508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:55.857269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-975b61b4-c059-4be7-8564-28ded595508b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:55.858498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-975b61b4-c059-4be7-8564-28ded595508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:55.932779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 27/48] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:55.956072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cced58d-2373-47e3-91c9-fec48b8664f2 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.021065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:56.067042 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cced58d-2373-47e3-91c9-fec48b8664f2 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.102242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece945-4ce9-4599-9a1e-93f906439af9 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.102533 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece945-4ce9-4599-9a1e-93f906439af9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.102705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.102842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece945-4ce9-4599-9a1e-93f906439af9 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.103514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.103795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.104066 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.104165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 28/49] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:56.109324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.109611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.109889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.114557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:33:56.122408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:33:56.123444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:33:56.124614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 22/50] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:33:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:33:56.127385 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:56.128580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:56.129348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.208014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 29/51] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:56.230903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.237750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce557-1f28-494d-874b-5c533ddbc65f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.253748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:56.258135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:56.272956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.273975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.274442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.275382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.276647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:56.277188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:56.279746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:56.290685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 23/52] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:56.296206 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.334264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447251-c38c-4347-8ae1-fcce3624ace8 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.368665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.369429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.369705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.369983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.371504 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:56.373022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:56.374693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 30/53] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:56.380339 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.395496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ea3e-b5dc-47aa-aaca-291a3140794f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.428492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.429462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.429743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.430018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.431231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:56.432706 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:56.434390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 24/54] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:33:56 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:56.439787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.471988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:56.479119 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed5da3-b4ff-4bda-bc40-f60942366087 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.486743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:56.515560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.516270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.516550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.516844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.517589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:56.519085 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:56.543896 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d8a7-f2dd-4b94-9492-7802f625e2ef None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.551985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 31/55] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 14:33:56 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:33:56.582878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ae72a-672a-4ce1-bb2f-2df81bcd8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.583717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ae72a-672a-4ce1-bb2f-2df81bcd8825 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.584006 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ae72a-672a-4ce1-bb2f-2df81bcd8825 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.584254 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ae72a-672a-4ce1-bb2f-2df81bcd8825 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.585702 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ae72a-672a-4ce1-bb2f-2df81bcd8825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:56.587199 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ae72a-672a-4ce1-bb2f-2df81bcd8825 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:56.588917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 25/56] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:56.595126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:56.702680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54703-dcba-4dfb-93f5-1a2887b3a47a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:56.738420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:56.739101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:56.739365 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:56.739640 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:56.740475 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.domain.id=default, user.id=0a3ad0c94b4942efa0beedd41caa0934) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:33:56.740786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:56.742427 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bc158-ad3a-4843-8fff-1a1ea57afd60 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:56.749644 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 32/57] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 14:33:56 2022] GET /identity/v3/role_assignments?user.id=0a3ad0c94b4942efa0beedd41caa0934&scope.domain.id=default&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:33:58.762085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624197-a812-4aaf-b113-b5bf4ad099f3 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:58.762986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624197-a812-4aaf-b113-b5bf4ad099f3 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:58.762986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624197-a812-4aaf-b113-b5bf4ad099f3 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:58.765297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 26/58] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:33:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:33:58.771654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:58.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:58.772518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:58.792939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:33:58.794172 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:33:58.797266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:58.867474 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 33/59] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:33:58.904306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f624197-a812-4aaf-b113-b5bf4ad099f3 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:59.027384 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f624197-a812-4aaf-b113-b5bf4ad099f3 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:59.041080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:59.073624 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249267ac-4985-4d07-a709-1e5b14620c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:59.074401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249267ac-4985-4d07-a709-1e5b14620c7f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:59.074690 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249267ac-4985-4d07-a709-1e5b14620c7f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:59.074971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249267ac-4985-4d07-a709-1e5b14620c7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:59.076230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249267ac-4985-4d07-a709-1e5b14620c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:59.077732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249267ac-4985-4d07-a709-1e5b14620c7f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:59.079343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 27/60] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:33:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:33:59.085009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:33:59.199996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b0e2-55d6-4d67-9c37-b1b4b7cb9bb4 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:33:59.244804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a7bf3-ea45-40fd-9dba-9f2133dc03e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:33:59.245507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a7bf3-ea45-40fd-9dba-9f2133dc03e8 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:33:59.245792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a7bf3-ea45-40fd-9dba-9f2133dc03e8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:33:59.246023 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a7bf3-ea45-40fd-9dba-9f2133dc03e8 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:33:59.246787 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a7bf3-ea45-40fd-9dba-9f2133dc03e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:33:59.248374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a7bf3-ea45-40fd-9dba-9f2133dc03e8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:33:59.263537 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:33:59.282300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 34/61] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:33:59 2022] POST /identity/v3/users => generated 275 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:34:02.627323 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eb118-b2ce-4aec-837a-0312f8b89dd0 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.628341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eb118-b2ce-4aec-837a-0312f8b89dd0 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.628867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eb118-b2ce-4aec-837a-0312f8b89dd0 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:02.630487 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 28/62] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:02.636215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.636638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.636766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:02.662269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:02.663845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:02.699185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f7bc2-0f66-4107-8e0a-dac3e9a78de5 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.699531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f7bc2-0f66-4107-8e0a-dac3e9a78de5 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.699748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f7bc2-0f66-4107-8e0a-dac3e9a78de5 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:02.701357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 29/63] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:02.706860 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.707422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.707932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:02.742469 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:02.743843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:02.751900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 35/64] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:02.818856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:02.851806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 30/65] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:02.886094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:02.931021 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae19988-4546-4262-8c84-1d281277d05b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:02.959261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:02.968479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc2544-c745-49ea-881c-30b916da6ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:02.969332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc2544-c745-49ea-881c-30b916da6ff9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.969618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc2544-c745-49ea-881c-30b916da6ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.969896 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc2544-c745-49ea-881c-30b916da6ff9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:02.981360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc2544-c745-49ea-881c-30b916da6ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-25dc2544-c745-49ea-881c-30b916da6ff9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:02.989217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:02.990246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 36/66] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:02.994756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d000-f1ea-4eca-ada6-6c95489a21a0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.997390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d000-f1ea-4eca-ada6-6c95489a21a0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.997390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d000-f1ea-4eca-ada6-6c95489a21a0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:02.997390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 37/67] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:02.999008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:02.999300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:02.999582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.018645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:03.019952 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:03.035818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432db74-fd96-4b00-a811-80afc85cb7a4 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.057410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.077968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.079274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.079806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.080368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.092486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:03.094131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 38/68] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:03.095006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:03.101401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:03.102397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:03.103309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:03.105803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 31/69] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:03.112133 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.217577 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ece8c-cbd1-4a00-b1cd-de205b899610 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.235504 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.238172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01c65-1e32-4437-93d7-dfd3068ad8b4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.256753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.257495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.257902 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.258180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.258938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:03.259252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:03.260779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:03.271614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.272288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.272567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.272870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.273662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:03.274198 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:03.274317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 39/70] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:03.275660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aaa0f-6599-4b18-8abd-a24e67f024bb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:03.279613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.285563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 32/71] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:03.286965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1c12-a850-49f8-832f-4e4c6e37e39e None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.287420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1c12-a850-49f8-832f-4e4c6e37e39e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.287843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1c12-a850-49f8-832f-4e4c6e37e39e None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.289286 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 33/72] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:03.294170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1c12-a850-49f8-832f-4e4c6e37e39e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.330293 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.392283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33985-866f-4222-b006-dc2c276e3e39 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.405008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1c12-a850-49f8-832f-4e4c6e37e39e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.429379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.430210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.430493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.430776 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.438340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695b2da-f8d4-4bc3-82a0-df1caaeca566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.439271 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695b2da-f8d4-4bc3-82a0-df1caaeca566 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.439552 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695b2da-f8d4-4bc3-82a0-df1caaeca566 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.439826 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695b2da-f8d4-4bc3-82a0-df1caaeca566 None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.441231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:03.449394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:03.450396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:03.451294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 40/73] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:03.452166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.459215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695b2da-f8d4-4bc3-82a0-df1caaeca566 None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:03.460827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695b2da-f8d4-4bc3-82a0-df1caaeca566 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:03.462757 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 34/74] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:03.463798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714de826-c7b5-48f9-aea9-395e90c3db30 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.464038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714de826-c7b5-48f9-aea9-395e90c3db30 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.464308 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714de826-c7b5-48f9-aea9-395e90c3db30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.480194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-714de826-c7b5-48f9-aea9-395e90c3db30 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:03.481414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-714de826-c7b5-48f9-aea9-395e90c3db30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:03.493965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.553515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 35/75] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:03.554849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399bda5-d379-4d71-99cc-b40d52af093f None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.555136 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399bda5-d379-4d71-99cc-b40d52af093f None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.555410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399bda5-d379-4d71-99cc-b40d52af093f None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.556676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 36/76] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:03.561131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399bda5-d379-4d71-99cc-b40d52af093f None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.566985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2084bd-f119-4cc3-ae21-97be2e916b3b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.593810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.601828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.602726 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.603014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.603295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.604503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:03.606044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:03.607700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 41/77] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:03.610090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.674163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399bda5-d379-4d71-99cc-b40d52af093f None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.709071 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.709778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.710059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.710178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd02df-11b8-47e9-a577-d9e3154e1d78 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.710642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.711059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:03.711389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:03.712756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8fba9-ab50-4215-8370-cfdf0baa0aaf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:03.717097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.721260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 37/78] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:03.722448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.722732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.723001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.739166 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:03.740921 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:03.751856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.752522 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.752808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.753087 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.806975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 38/79] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:03.807805 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.824964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.832012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:03.834000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:03.909145 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a17d-9932-47c5-aaf0-d3a1e4c4b001 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:03.934356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 42/80] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:03.935400 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:03.943743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:03.944369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:03.945187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:03.945500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:03.945772 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:03.956408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:03.964328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:03.965478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 39/81] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:03.966908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.039989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ccce3-0823-4ae2-a064-4268008db66c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.041967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.068365 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c10a8-651b-4c5c-bb6b-65c71d8cd98d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.073812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.074471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.074753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.075031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.102390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.103273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.103556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.103827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.105130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.106668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.108419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 40/82] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.110792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.148800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.150692 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.158329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.220245 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e09d5c-6bcf-4e61-a695-8a325e56ecff None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.253477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.253879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 43/83] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:04.254289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.254574 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.254852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.256068 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.257647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.259283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 41/84] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.259524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.260048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.272426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.363421 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22db771-7a30-4386-a9af-364f05f0318b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.364390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c3713-2324-4cf3-a5c1-9d8002a11771 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.373204 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.398811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.399790 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.400073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.400349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.401111 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:04.401446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.402857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.404969 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.405883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.406169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.406448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.407652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.409101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.420047 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 44/85] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.420217 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 42/86] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.420217 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.420443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.481271 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1431451-ab32-4e7f-813b-bdd9ecb03f90 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.518008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb16d-5eba-4b37-a041-5b6afd38475a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.548432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.549344 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.549634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.549912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.551087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.552415 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.552538 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.553076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.553397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.553679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.554373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 43/87] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.555124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.599579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.624208 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.625846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.655742 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaf5f7-4294-4038-b02a-8561328b4a5c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.689026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.689740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.690024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.690302 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.691014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.692475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.707412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.719557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 44/88] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:04 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:04.720368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.733343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 45/89] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:04.733343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.813459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:04.836994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643dca-f1e1-446a-8d40-a3ae5927e7d5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.841360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef502e-e5e9-4b84-bc98-3fbdab38cbd1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:04.874193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.874902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.875230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.875736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.876404 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=266c22fce804450d859a7e09765df901, user.id=9992281c74bc46bab68fdf142522a8a7) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:04.876725 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.878278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.880645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:04.881687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:04.882033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:04.882315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:04.883635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:04.884847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 45/90] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/role_assignments?user.id=9992281c74bc46bab68fdf142522a8a7&scope.project.id=266c22fce804450d859a7e09765df901&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.885170 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:04.886915 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 46/91] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:04.891532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.895114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:04.976727 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:05.025438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b188b2-75c9-4312-a8a5-cc12fa1981ce None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:05.029877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d00543-4b18-4dda-8aa8-7c158b654185 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:05.072095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:05.073512 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:05.073763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:05.074189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:05.074587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:05.074883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:05.074990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:05.075279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:05.076198 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=266c22fce804450d859a7e09765df901, user.id=0a3ad0c94b4942efa0beedd41caa0934) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:05.076519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:05.077045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:05.078786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:05.079097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:05.081480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 47/92] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:05.082470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:05.086235 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 46/93] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:04 2022] GET /identity/v3/role_assignments?user.id=0a3ad0c94b4942efa0beedd41caa0934&scope.project.id=266c22fce804450d859a7e09765df901&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:05.088018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:05.103012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:05.207249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f466d00-a8ad-4029-b955-e79a0bc7d84a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:05.218926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79b6c-a164-4796-86f3-8c640f119ce9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:05.250219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:05.250863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:05.252523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:05.254050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:05.254347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:05.265858 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:05.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:05.267810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:05.268239 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:05.269275 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] RBAC: Adding query filter params (role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=266c22fce804450d859a7e09765df901, user.id=9992281c74bc46bab68fdf142522a8a7) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:05.270253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:05.272229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:05.280211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 47/94] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:05 2022] GET /identity/v3/role_assignments?user.id=9992281c74bc46bab68fdf142522a8a7&scope.project.id=266c22fce804450d859a7e09765df901&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:05.376542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:05.378149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:05.391536 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:05.503602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 48/95] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:05.509632 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:05.614919 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fccdc-dffe-4855-a26d-1bd5c5e2faee None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:05.659179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9575436-bd3f-4c76-83bc-2c703433b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:05.660067 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9575436-bd3f-4c76-83bc-2c703433b27d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:05.660350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9575436-bd3f-4c76-83bc-2c703433b27d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:05.660624 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9575436-bd3f-4c76-83bc-2c703433b27d None admin] PATH_INFO: `/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:05.661878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9575436-bd3f-4c76-83bc-2c703433b27d None admin] RBAC: Authorizing `identity:get_project(project_id=0b6ccab92dcf4632a87c8f5bdac5a63d)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:05.663330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9575436-bd3f-4c76-83bc-2c703433b27d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:05.665080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 48/96] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:05 2022] GET /identity/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:05.671111 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:05.799006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e770cc-790c-4278-8420-6e44138d0954 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:05.812791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:05.839163 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:05.839842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:05.840126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:05.840393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:05.841232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=0b6ccab92dcf4632a87c8f5bdac5a63d, user.id=9992281c74bc46bab68fdf142522a8a7) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:05.841569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:05.843078 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b457b0b-3a6a-481f-bb9f-9fb241c44a13 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:05.849021 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 49/97] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:05 2022] GET /identity/v3/role_assignments?user.id=9992281c74bc46bab68fdf142522a8a7&scope.project.id=0b6ccab92dcf4632a87c8f5bdac5a63d&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:07.896815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d21ea-d460-41c6-84d9-cc6001dca4b8 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:07.897398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d21ea-d460-41c6-84d9-cc6001dca4b8 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:07.897398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d21ea-d460-41c6-84d9-cc6001dca4b8 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:07.898883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 49/98] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:07.914958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16600ca3-fd8a-48d2-8742-00086e9387d7 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:07.915420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16600ca3-fd8a-48d2-8742-00086e9387d7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:07.915524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16600ca3-fd8a-48d2-8742-00086e9387d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:07.941349 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-16600ca3-fd8a-48d2-8742-00086e9387d7 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:07.942548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-16600ca3-fd8a-48d2-8742-00086e9387d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:08.042966 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 50/99] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:08.081643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d21ea-d460-41c6-84d9-cc6001dca4b8 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.157883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:08.166409 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af629b-74d7-4525-a898-7f491eb9b6fd None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.166694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af629b-74d7-4525-a898-7f491eb9b6fd None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.167019 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af629b-74d7-4525-a898-7f491eb9b6fd None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.168423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 51/100] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:08.174638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.174946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.175237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.202709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:08.203934 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:08.255380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d21ea-d460-41c6-84d9-cc6001dca4b8 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:08.314982 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 52/101] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:08.320158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:08.320978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.321282 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.321566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.332612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:08.340514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:08.341570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:08.342612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 50/102] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:08.349276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.349683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.404737 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:08.465350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848893be-62c8-4113-8d26-20b3bb4e0f0f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:08.485264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:08.505266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:08.505266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.505266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.505266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.514488 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974e35-770f-48dc-88ea-c47c33904143 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:08.520665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:08.528996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:08.530025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:08.530970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 51/103] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:08.541262 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.578878 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:08.580033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.580524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.581022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.582006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:08.582545 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:08.584191 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:08.597009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 53/104] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:08.604962 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.679579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5596b-d23b-479b-8660-bb36a3b93bbf None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:08.738808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:08.739806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.740291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.740769 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.742190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:08.742822 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:08.745222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:08.762045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 52/105] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:08.770977 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.801286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:08.810837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a645a36-1ce4-4a15-bcab-472f7a0f1ed6 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:08.855750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:08.899196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:08.901483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:08.902231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:08.902957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:08.909165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:08.912144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:08.915490 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 54/106] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:08.934003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:08.970069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fef59e-608a-4ca2-94dc-524e8ad247d0 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.037403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.037403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.037403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.037403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.038088 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:09.040128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:09.043195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 53/107] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:09.051675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.140601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a914f5-3055-4338-ae0a-4032bee2f02d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.181171 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf83238-9fff-4845-9e28-0a211e6cc4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.182109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf83238-9fff-4845-9e28-0a211e6cc4c2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.182394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf83238-9fff-4845-9e28-0a211e6cc4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.182672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf83238-9fff-4845-9e28-0a211e6cc4c2 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.183859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf83238-9fff-4845-9e28-0a211e6cc4c2 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:09.185397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf83238-9fff-4845-9e28-0a211e6cc4c2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:09.187121 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 55/108] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:08 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:09.188411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed5e3b-a033-407c-8613-28137bdceb61 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.188702 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed5e3b-a033-407c-8613-28137bdceb61 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.188981 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed5e3b-a033-407c-8613-28137bdceb61 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.190488 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 56/109] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:09.195773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a71531-905c-489b-9399-f8eca51a107d None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.196074 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a71531-905c-489b-9399-f8eca51a107d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.196363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a71531-905c-489b-9399-f8eca51a107d None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.197838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 57/110] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:09.198861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f329b2-553d-4537-a427-4e8d272dcf61 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.199220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f329b2-553d-4537-a427-4e8d272dcf61 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.199509 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f329b2-553d-4537-a427-4e8d272dcf61 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.200215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a855087-56c1-4d67-939e-8e3115294bd4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.200733 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 58/111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:09.201442 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f329b2-553d-4537-a427-4e8d272dcf61 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.210576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:09.237518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c8262-24aa-4f5a-9896-0f5b4a7df07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.238518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c8262-24aa-4f5a-9896-0f5b4a7df07f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.238793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c8262-24aa-4f5a-9896-0f5b4a7df07f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.239044 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c8262-24aa-4f5a-9896-0f5b4a7df07f None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.240248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c8262-24aa-4f5a-9896-0f5b4a7df07f None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:09.241409 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:09.241814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c8262-24aa-4f5a-9896-0f5b4a7df07f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:09.243546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 54/112] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:09 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:09.244813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f58b3-da92-44b5-8912-854d340a9eb5 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.245110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f58b3-da92-44b5-8912-854d340a9eb5 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.245447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f58b3-da92-44b5-8912-854d340a9eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.265407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-908f58b3-da92-44b5-8912-854d340a9eb5 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:09.266807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-908f58b3-da92-44b5-8912-854d340a9eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:09.320062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f329b2-553d-4537-a427-4e8d272dcf61 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.336738 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 55/113] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:09.339965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.339965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.339965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.354329 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:09.355567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:09.360230 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e566b-c87e-48a2-99ad-53f3e347856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.361136 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e566b-c87e-48a2-99ad-53f3e347856c None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.361447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e566b-c87e-48a2-99ad-53f3e347856c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.361722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e566b-c87e-48a2-99ad-53f3e347856c None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901/users/9992281c74bc46bab68fdf142522a8a7/roles/6b466334835d44468e51d8a0bfed2e86` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.363753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e566b-c87e-48a2-99ad-53f3e347856c None admin] RBAC: Authorizing `identity:create_grant(project_id=266c22fce804450d859a7e09765df901, user_id=9992281c74bc46bab68fdf142522a8a7, role_id=6b466334835d44468e51d8a0bfed2e86)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:09.365281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e566b-c87e-48a2-99ad-53f3e347856c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:09.378666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 59/114] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:09 2022] PUT /identity/v3/projects/266c22fce804450d859a7e09765df901/users/9992281c74bc46bab68fdf142522a8a7/roles/6b466334835d44468e51d8a0bfed2e86 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:09.379963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe12e-5d8a-4f12-b8da-4a1cf5dac0ad None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.380189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe12e-5d8a-4f12-b8da-4a1cf5dac0ad None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.380478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe12e-5d8a-4f12-b8da-4a1cf5dac0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.401925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4febe12e-5d8a-4f12-b8da-4a1cf5dac0ad None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:09.403539 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4febe12e-5d8a-4f12-b8da-4a1cf5dac0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:09.430112 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 56/115] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:09.434791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.482896 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:09.483279 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 60/116] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:09.485681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc410a34-1a27-4832-8229-e4f3648803e0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.486058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc410a34-1a27-4832-8229-e4f3648803e0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.486303 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc410a34-1a27-4832-8229-e4f3648803e0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.492262 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 61/117] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:09.492262 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc410a34-1a27-4832-8229-e4f3648803e0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.526125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:09.565803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc3af3-58f5-48b1-b832-457f19e9f4d0 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.613578 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc410a34-1a27-4832-8229-e4f3648803e0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.614174 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.615099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.615377 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.615650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901/users/0a3ad0c94b4942efa0beedd41caa0934/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.617774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] RBAC: Authorizing `identity:create_grant(project_id=266c22fce804450d859a7e09765df901, user_id=0a3ad0c94b4942efa0beedd41caa0934, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:09.619245 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:09.635440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 57/118] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:09 2022] PUT /identity/v3/projects/266c22fce804450d859a7e09765df901/users/0a3ad0c94b4942efa0beedd41caa0934/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:09.636313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.659107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b877909-8dab-4889-aac2-3e1bc4549243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.660593 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877909-8dab-4889-aac2-3e1bc4549243 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.660882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877909-8dab-4889-aac2-3e1bc4549243 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.661159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877909-8dab-4889-aac2-3e1bc4549243 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.674439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b877909-8dab-4889-aac2-3e1bc4549243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-1b877909-8dab-4889-aac2-3e1bc4549243 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:09.683309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:09.684476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:09.685496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 62/119] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:09.686341 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.686341 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.686497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.705762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:09.707196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:09.760635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:09.771906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba200-88f4-459f-b984-db991cee4340 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.797290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 63/120] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:09.797290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.807266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:09.835332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.836522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.836985 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.837488 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.853933 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:09.862606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:09.863798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:09.864719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:09.867030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 58/121] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:09.868177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.902793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706d3e8-e0ca-43e3-a27c-882ea4cb587e None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:09.943076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:09.944129 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:09.944542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:09.945089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:09.958467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:09.969900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:09.970946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:09.971876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 64/122] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:09.972637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:09.992908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60954e-9d44-4d5c-a263-e0dd1ec1e21a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.036318 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.037365 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.038493 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.038770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.039559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:10.039874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.041302 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.050117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 59/123] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.050953 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.069668 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:10.070052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:10.100264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cc5d7-ed4d-4da5-acbe-6241b6ab05d4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.145751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.146638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.147016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.147386 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.148792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.150714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.152468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 65/124] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.157788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.164505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf860469-cf8f-4697-8cf7-33483412f31f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.216005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.217405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.217872 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.218316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.219241 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:10.219554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.221902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.233091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 60/125] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.233963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.302375 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ca67f-0cb5-4aed-b456-ed763dae0593 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.323091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:10.346700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.347385 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.347673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.348139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.348728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:10.349129 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.350516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.351660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a194ec-8008-414e-b822-f7e3c02a5333 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.363831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 66/126] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.363831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.366584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:10.398980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.401213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.402005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.402510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.404758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.406844 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.409607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 61/127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.410621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.484090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515f2dd-9a79-4b54-a67e-3f7ca10eb4cc None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.528166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.528846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.529127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.529565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.530278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.531759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.552988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f993-fcc2-4d65-8695-d706ae682616 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.562058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 67/128] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:34:10 2022] POST /identity/v3/projects => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:10.566263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.624688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.625651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.625940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.626222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] PATH_INFO: `/v3/users/6e952459c8a44e96a841dc6915e0acdb` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.656058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] RBAC: Authorizing `identity:get_user(user_id=6e952459c8a44e96a841dc6915e0acdb)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.657524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.659550 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 62/129] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/users/6e952459c8a44e96a841dc6915e0acdb => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.660489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.669636 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:10.687278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:10.779979 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c47d3d-5297-4394-b55b-7a2c23b4277e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.841339 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.842273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.842557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.842829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.844072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.845767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.848479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 68/130] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.849373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.864034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfcc9e-d0b5-4e1d-b8f8-f7e98811b921 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.905677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:10.906669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:10.906876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:10.907159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] PATH_INFO: `/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:10.908376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=0b6ccab92dcf4632a87c8f5bdac5a63d)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:10.909899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:10.911809 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 63/131] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:10.914707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:10.971637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dc2a6-4ed7-4d49-8553-0b42f8001633 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:10.996116 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:11.001694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:11.011379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:11.013218 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.013530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.013802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.033267 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d05323-1f45-4fd2-ab8f-900ca7d2c5e2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:11.071679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:11.073074 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.073732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.074222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.080151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:11.082085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:11.084196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 64/132] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:11.085390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:11.092527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:11.094092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:11.202569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 69/133] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:11.202569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:11.224580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:11.265713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382c2b-4454-4979-b34d-b5de312540cd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:11.303745 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:11.305079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.305747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.306277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] PATH_INFO: `/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d/users/9992281c74bc46bab68fdf142522a8a7/roles/6b466334835d44468e51d8a0bfed2e86` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.310756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b6ccab92dcf4632a87c8f5bdac5a63d, user_id=9992281c74bc46bab68fdf142522a8a7, role_id=6b466334835d44468e51d8a0bfed2e86)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:11.312596 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:11.328099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 65/134] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:11 2022] PUT /identity/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d/users/9992281c74bc46bab68fdf142522a8a7/roles/6b466334835d44468e51d8a0bfed2e86 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:11.328680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:11.330207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:11.401133 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdb13b-26f3-4314-905c-73af0eea0832 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:11.443423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:11.444349 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.444663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.444944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901/users/9992281c74bc46bab68fdf142522a8a7/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.447406 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] RBAC: Authorizing `identity:create_grant(project_id=266c22fce804450d859a7e09765df901, user_id=9992281c74bc46bab68fdf142522a8a7, role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:11.448905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:11.464399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 70/135] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:11 2022] PUT /identity/v3/projects/266c22fce804450d859a7e09765df901/users/9992281c74bc46bab68fdf142522a8a7/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:11.533978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa0f3-ea06-4213-9306-7a75ce4372d6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:11.575513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92204395-5828-4886-a5a9-4fc4829c462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:11.576410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92204395-5828-4886-a5a9-4fc4829c462a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.576691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92204395-5828-4886-a5a9-4fc4829c462a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.576970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92204395-5828-4886-a5a9-4fc4829c462a None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.578272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92204395-5828-4886-a5a9-4fc4829c462a None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:11.579782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92204395-5828-4886-a5a9-4fc4829c462a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:11.581662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 66/136] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:11 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:11.588192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:11.723280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:11.746271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb22a8-c53c-446e-b388-b7aad8a02950 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:11.785043 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:11.785818 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.786102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.786425 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.787280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] RBAC: Adding query filter params (role.id=82a463123e20467bb4c64ad6789ecfc8, scope.project.id=266c22fce804450d859a7e09765df901, user.id=6e952459c8a44e96a841dc6915e0acdb) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:11.787592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:11.789153 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91300f4d-0b75-4fc8-99bf-a60511e8e148 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:11.795730 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 71/137] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:11 2022] GET /identity/v3/role_assignments?user.id=6e952459c8a44e96a841dc6915e0acdb&scope.project.id=266c22fce804450d859a7e09765df901&role.id=82a463123e20467bb4c64ad6789ecfc8 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:11.878440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068fbcb-49b8-4615-ad1e-921e08e00f7d None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.879135 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068fbcb-49b8-4615-ad1e-921e08e00f7d None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.879625 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068fbcb-49b8-4615-ad1e-921e08e00f7d None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.881317 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 67/138] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:11.887446 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e729-4554-421a-a0ef-c320a104f74e None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:11.887806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e729-4554-421a-a0ef-c320a104f74e None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:11.888088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e729-4554-421a-a0ef-c320a104f74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:11.906646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-9ea6e729-4554-421a-a0ef-c320a104f74e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:11.907899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-9ea6e729-4554-421a-a0ef-c320a104f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:11.981697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 72/139] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:12.005518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068fbcb-49b8-4615-ad1e-921e08e00f7d None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:12.096247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:12.142094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068fbcb-49b8-4615-ad1e-921e08e00f7d None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:12.187899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:12.188783 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.189071 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.189372 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.206798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:12.215123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:12.216148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:12.217054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:12.221269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764cb2d-1af7-4029-ae0c-90d1714b2727 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.221269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764cb2d-1af7-4029-ae0c-90d1714b2727 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.221269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764cb2d-1af7-4029-ae0c-90d1714b2727 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.221550 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 68/140] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:12.221644 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 73/141] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:12.226347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:12.230734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.231021 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.231306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.265752 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:12.266861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:12.345424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c80834-eac3-4c13-be89-982a1b8cce55 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:12.372751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:12.383070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:12.383761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.384042 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.384314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.385073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:12.385769 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:12.387997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:12.405462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 69/142] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:12.413951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:12.420417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 74/143] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:12.456202 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:12.464723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:12.576717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8549bc-92e9-4570-809b-7dda6794fd4a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:12.613095 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:12.614018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.614298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.614567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.615771 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:12.617308 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:12.619028 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 70/144] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:12.624646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:12.634232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bccc-f65f-4f8f-a5b5-7eea0abfb5b7 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:12.669718 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:12.670542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.670828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.671102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.681988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:12.685667 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:12.689897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:12.690898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:12.692695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 75/145] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:12.696925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:12.730108 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7d6af-315f-452c-a5b1-48d82cf3cff6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:12.813700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:12.821705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e67c0-e5ff-4ccc-92cf-21055905296b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:12.866619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:12.867437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.871752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.871752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.886708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:12.887547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:12.888010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:12.888481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:12.889969 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:12.890474 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:12.892630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:12.927642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 76/146] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:12.937686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:13.055026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:13.056554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:13.133409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:13.219422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 71/147] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:34:13.226501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:13.227222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f140-d988-4b19-b924-923c0e366c0a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:13.296278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:13.312007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:13.312900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:13.313176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:13.313471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] PATH_INFO: `/v3/users/0a3ad0c94b4942efa0beedd41caa0934` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:13.317689 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] RBAC: Authorizing `identity:get_user(user_id=0a3ad0c94b4942efa0beedd41caa0934)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:13.319204 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:13.321182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 77/148] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:12 2022] GET /identity/v3/users/0a3ad0c94b4942efa0beedd41caa0934 => generated 247 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:13.327132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:13.403209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873a00-2c22-4aed-819c-9a1bf69546f8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:13.466066 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:13.466943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:13.467215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:13.467481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:13.470438 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:13.471973 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:13.473893 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 72/149] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:13 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:13.482077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:13.515933 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8f1c5-ceee-403f-bc23-981649a4447d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:13.531902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:13.576482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:13.577155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:13.577464 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:13.577731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:13.662642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce43f-9142-42b1-8972-c8cd2870f868 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:13.691742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:13.707163 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:13.708677 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:13.730123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:13.730799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:13.731077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:13.731342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:13.732250 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=266c22fce804450d859a7e09765df901, user.id=9992281c74bc46bab68fdf142522a8a7) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:13.732556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:13.734027 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:13.748876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 73/150] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:13 2022] GET /identity/v3/role_assignments?user.id=9992281c74bc46bab68fdf142522a8a7&scope.project.id=266c22fce804450d859a7e09765df901&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:13.857482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 78/151] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:34:13.862825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:14.004390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:14.014859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a8f1-de63-4d62-9d75-418449014edf None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:14.066912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a126a5-af07-4d3f-bf30-e699e2d191cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:14.067798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a126a5-af07-4d3f-bf30-e699e2d191cc None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.068076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a126a5-af07-4d3f-bf30-e699e2d191cc None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.068345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a126a5-af07-4d3f-bf30-e699e2d191cc None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.069705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a126a5-af07-4d3f-bf30-e699e2d191cc None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:14.071179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a126a5-af07-4d3f-bf30-e699e2d191cc None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:14.073287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 74/152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:13 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:14.078828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:14.182373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdabb85-4f6b-43ff-8db5-78e575877994 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.182731 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdabb85-4f6b-43ff-8db5-78e575877994 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.182934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdabb85-4f6b-43ff-8db5-78e575877994 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.184479 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 75/153] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:14.189866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.190461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.190945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.216881 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:14.218614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:14.221482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:14.227940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3cbec-1b63-4f5a-9c2c-471d72108943 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:14.268459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:14.269054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.269367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.269643 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.270486 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=266c22fce804450d859a7e09765df901, user.id=0a3ad0c94b4942efa0beedd41caa0934) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:14.270798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:14.272270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:14.280367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 79/154] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:14 2022] GET /identity/v3/role_assignments?user.id=0a3ad0c94b4942efa0beedd41caa0934&scope.project.id=266c22fce804450d859a7e09765df901&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:14.314878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 76/155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:14.342143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:14.465628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1b2c7-3523-4457-b2cf-46508f7931ec None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:14.513307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714185b-cc46-405d-a6cf-9c716811ba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:14.514158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714185b-cc46-405d-a6cf-9c716811ba0d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.514440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714185b-cc46-405d-a6cf-9c716811ba0d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.515218 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714185b-cc46-405d-a6cf-9c716811ba0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.518023 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714185b-cc46-405d-a6cf-9c716811ba0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:14.519794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714185b-cc46-405d-a6cf-9c716811ba0d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:14.521769 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 80/156] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:14.527952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:14.675470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:14.693804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa41b2-e958-4dfd-a404-986721e3e3b8 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:14.740579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df0ca9-5508-46b4-b465-02797a4ece6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:14.741435 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0ca9-5508-46b4-b465-02797a4ece6d None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.741721 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0ca9-5508-46b4-b465-02797a4ece6d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.741992 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df0ca9-5508-46b4-b465-02797a4ece6d None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.742722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df0ca9-5508-46b4-b465-02797a4ece6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:14.744224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df0ca9-5508-46b4-b465-02797a4ece6d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:14.771446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 77/157] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:14 2022] POST /identity/v3/users => generated 283 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:14.920040 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ce470-4c5d-4de1-ab49-f5c62556377f None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.920487 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ce470-4c5d-4de1-ab49-f5c62556377f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.920597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ce470-4c5d-4de1-ab49-f5c62556377f None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.922490 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 81/158] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:14.930775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:14.930775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:14.930775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:14.949966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:14.951193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:15.000393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5e2f4-a7e2-4b37-9e94-437c4a96e874 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.001037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5e2f4-a7e2-4b37-9e94-437c4a96e874 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.001446 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5e2f4-a7e2-4b37-9e94-437c4a96e874 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.003002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 82/159] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:15.008391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310390d4-8f97-40ec-87a2-fbe0bace5eda None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.008678 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310390d4-8f97-40ec-87a2-fbe0bace5eda None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.008950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310390d4-8f97-40ec-87a2-fbe0bace5eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.030673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-310390d4-8f97-40ec-87a2-fbe0bace5eda None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:15.032226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-310390d4-8f97-40ec-87a2-fbe0bace5eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:15.039704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 78/160] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:15.060443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:15.065895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.123643 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 83/161] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:15.125025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474d034-7ca7-443c-9fcd-2d184196f6f2 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.125348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474d034-7ca7-443c-9fcd-2d184196f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.125626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474d034-7ca7-443c-9fcd-2d184196f6f2 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.127036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 84/162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:15.132919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.133195 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.133502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.152361 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:15.153799 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:15.159850 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:15.194350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37486c88-a0e4-4395-bd64-37d42a9ebed1 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.231916 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.232722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.233167 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.233333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 85/163] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:15.237324 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.237424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.261161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:15.272430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:15.273565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:15.275023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:15.276114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 79/164] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:15.276856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.346630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:15.361102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e6c42-f640-4068-a5bf-981322ef8d5e None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.396741 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d772f-381f-4d5f-afc6-8f04e1c9b3af None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.407139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.407960 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.408241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.408745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.420480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:15.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:15.430088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:15.430998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:15.431702 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 86/165] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:15.432258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.441633 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.442544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.442894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.443173 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.445495 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:15.456834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:15.468938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:15.470064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:15.470991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 80/166] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:15.471759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.589765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9aa2ca-f7fa-40a7-9b1a-b9255d58af5b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.636418 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2b7ae-8b84-406d-88f0-3618980ae5ed None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.654069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.654797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.655151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.655429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.656187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:15.656493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:15.658072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:15.672381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.673060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.673370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.673645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.674389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:15.674699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:15.676096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:15.676853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 87/167] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:15.677847 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 81/168] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:15.690327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.701376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:15.768295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:15.794196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d3bda-8782-48b4-9427-9cfb38023625 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.816114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0cff0-9eb8-439e-8d64-896adba66240 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.829085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.830017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.830297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.830569 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.832049 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:15.833558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:15.835520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 82/169] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:15.836073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.867004 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:15.867699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:15.867980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:15.868255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:15.869011 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:15.869347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:15.870754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:15.886955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 88/170] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:15.887755 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:15.958468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f81182-3624-46e3-8082-9f7118e8ef23 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:15.970111 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:16.005620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:16.025791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:16.026679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:16.026952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:16.027219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] PATH_INFO: `/v3/users/6e952459c8a44e96a841dc6915e0acdb` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:16.030967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd1318-0230-41c0-9820-63e81db94947 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:16.032899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] RBAC: Authorizing `identity:get_user(user_id=6e952459c8a44e96a841dc6915e0acdb)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:16.035419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:16.037505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 83/171] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/users/6e952459c8a44e96a841dc6915e0acdb => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:16.038621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:16.103109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859884b-e013-4376-8755-62040d217b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:16.103768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859884b-e013-4376-8755-62040d217b39 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:16.104041 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859884b-e013-4376-8755-62040d217b39 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:16.104309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859884b-e013-4376-8755-62040d217b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:16.187219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855631-93b0-463a-8263-cbd3a01fa39f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:16.223121 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:16.224017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:16.224290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:16.224557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] PATH_INFO: `/v3/users/9992281c74bc46bab68fdf142522a8a7` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:16.226441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] RBAC: Authorizing `identity:get_user(user_id=9992281c74bc46bab68fdf142522a8a7)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:16.229750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:16.234210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 84/172] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:16 2022] GET /identity/v3/users/9992281c74bc46bab68fdf142522a8a7 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:16.234990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:16.281175 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859884b-e013-4376-8755-62040d217b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:16.282808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859884b-e013-4376-8755-62040d217b39 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:16.313692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:16.357948 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ce091-999d-4758-a366-62013a8fb36f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:16.411867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:16.412748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:16.413050 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:16.413386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:16.415605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:16.417067 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:16.420418 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 85/173] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:16 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:16.423683 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:16.436862 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:16.487657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 89/174] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:16.488429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859884b-e013-4376-8755-62040d217b39 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:16.618248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859884b-e013-4376-8755-62040d217b39 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:16.651387 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a82d5-6d8e-43fc-a8ce-1c216ee4669e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:16.674705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:16.675612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:16.675892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:16.676166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901/users/6e952459c8a44e96a841dc6915e0acdb/roles/82a463123e20467bb4c64ad6789ecfc8` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:16.679384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] RBAC: Authorizing `identity:create_grant(project_id=266c22fce804450d859a7e09765df901, user_id=6e952459c8a44e96a841dc6915e0acdb, role_id=82a463123e20467bb4c64ad6789ecfc8)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:16.681644 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:16.688489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:16.697895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:16.698568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:16.698849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:16.699100 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:16.707429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 90/175] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:16 2022] PUT /identity/v3/projects/266c22fce804450d859a7e09765df901/users/6e952459c8a44e96a841dc6915e0acdb/roles/82a463123e20467bb4c64ad6789ecfc8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:16.708058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:16.791067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:16.878362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:16.879886 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:16.929619 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46887f-eb73-42e3-b70e-6f6b3c7e827d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:17.007326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:17.008228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:17.008684 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:17.008963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] PATH_INFO: `/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:17.014109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] RBAC: Authorizing `identity:get_project(project_id=0b6ccab92dcf4632a87c8f5bdac5a63d)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:17.016476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:17.018516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 91/176] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:16 2022] GET /identity/v3/projects/0b6ccab92dcf4632a87c8f5bdac5a63d => generated 312 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:17.032898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:17.101955 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 86/177] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:17.110141 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:17.170840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:17.229629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:17.234534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685a33-15ed-4f1f-a4d6-4d26584772ea None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:17.310039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:17.311198 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:17.311493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:17.311771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:17.312623 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=0b6ccab92dcf4632a87c8f5bdac5a63d, user.id=9992281c74bc46bab68fdf142522a8a7) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:17.312997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:17.314554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:17.325860 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 92/178] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:17 2022] GET /identity/v3/role_assignments?user.id=9992281c74bc46bab68fdf142522a8a7&scope.project.id=0b6ccab92dcf4632a87c8f5bdac5a63d&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:17.351765 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025143c3-efe6-4021-ae86-bcff04333615 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:17.423991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6bcff-1784-4a9d-8403-edf2cd63904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:17.424798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6bcff-1784-4a9d-8403-edf2cd63904f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:17.425080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6bcff-1784-4a9d-8403-edf2cd63904f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:17.425383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6bcff-1784-4a9d-8403-edf2cd63904f None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:17.426643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6bcff-1784-4a9d-8403-edf2cd63904f None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:17.428132 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6bcff-1784-4a9d-8403-edf2cd63904f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:17.431146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 87/179] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:17 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:17.436065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:17.580742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb13de-8842-4141-b99d-80cfb8ad26cc None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:17.632966 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:17.633694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:17.633977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:17.634245 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:17.635099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] RBAC: Adding query filter params (role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=266c22fce804450d859a7e09765df901, user.id=9992281c74bc46bab68fdf142522a8a7) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:17.635405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:17.636985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035b709-44b0-4d35-a0fb-61c9029e84f9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:17.643867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 93/180] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:17 2022] GET /identity/v3/role_assignments?user.id=9992281c74bc46bab68fdf142522a8a7&scope.project.id=266c22fce804450d859a7e09765df901&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:17.669847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3340-57dd-47e6-9fea-87da05353c4d None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:17.670202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3340-57dd-47e6-9fea-87da05353c4d None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:17.670399 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3340-57dd-47e6-9fea-87da05353c4d None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:17.673022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 88/181] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:17.677953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:17.678261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:17.678514 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:17.701859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:17.703143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:17.734671 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:17.793467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 94/182] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:17.832681 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3340-57dd-47e6-9fea-87da05353c4d None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:17.969017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:17.977251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3340-57dd-47e6-9fea-87da05353c4d None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.021151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16901aa3-eb22-4eca-ba81-bfb5ee30e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.021999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16901aa3-eb22-4eca-ba81-bfb5ee30e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.022277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16901aa3-eb22-4eca-ba81-bfb5ee30e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.022578 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16901aa3-eb22-4eca-ba81-bfb5ee30e0a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.033955 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16901aa3-eb22-4eca-ba81-bfb5ee30e0a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-16901aa3-eb22-4eca-ba81-bfb5ee30e0a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:18.043778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:18.044869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:18.045855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:18.048865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 89/183] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:18.054402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.062807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04268a5-8ef9-4929-9f60-7a6820e7076a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.063106 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04268a5-8ef9-4929-9f60-7a6820e7076a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.063364 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04268a5-8ef9-4929-9f60-7a6820e7076a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.064950 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 90/184] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:18.070633 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75805bb1-0c09-45f4-bf7e-62ac0a155552 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.070906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75805bb1-0c09-45f4-bf7e-62ac0a155552 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.071186 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75805bb1-0c09-45f4-bf7e-62ac0a155552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.092844 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-75805bb1-0c09-45f4-bf7e-62ac0a155552 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:18.094477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-75805bb1-0c09-45f4-bf7e-62ac0a155552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:18.183965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 91/185] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:18.184355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc608-74d8-4410-a553-6608dacc26b7 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.184608 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc608-74d8-4410-a553-6608dacc26b7 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.184889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc608-74d8-4410-a553-6608dacc26b7 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.187011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 92/186] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:18.193224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.197524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.197524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.202260 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5eb5b-9ccc-4c0b-8770-7c05ce89d77d None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.220097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:18.221172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:18.251769 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.252617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.252905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.253208 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.254027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:18.254374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:18.255797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:18.265549 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 95/187] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:18.270750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.290587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:18.306543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 93/188] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:18.306543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.325571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:18.382454 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ad82f-e978-4bf5-897e-1643a79599ce None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.408590 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f34c4-8656-4785-b36f-eeaa9a27e1bf None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.419343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.420255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.420645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.420946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.431298 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:18.439601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:18.440613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 96/189] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:18.441831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.450344 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.451527 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.451979 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.452396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] PATH_INFO: `/v3/users/2e58ecc0cf7640b59fba11a624e095d8` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.468799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] RBAC: Authorizing `identity:get_user(user_id=2e58ecc0cf7640b59fba11a624e095d8)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:18.471003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:18.472906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 94/190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/users/2e58ecc0cf7640b59fba11a624e095d8 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:18.473620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.544692 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:18.561553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:18.569106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73dc92-c9ed-40cb-bdd3-43412d2ac440 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.589226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d61c7-86a2-48a4-b53e-bd45b144af2b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.604111 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.604917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.605203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.605621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.606769 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:18.608336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:18.610210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 97/191] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:18.615300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.627106 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.627809 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.628194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.628466 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.631094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:18.631877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:18.634113 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:18.650102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 95/192] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:18.650102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.724070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f4b7-5bbe-4e61-9379-d023cfd7b54f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.750750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c0a06-a0f4-43fb-a615-d6b0e5a87b75 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.759854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.760838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.761263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.761552 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.780828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:18.785030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:18.785736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:18.786014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:18.786287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:18.787010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:18.788545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:18.796775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:18.812782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 96/193] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:18 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:18.822410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:18.840985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:18.843067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:18.967586 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d27ad-93a4-411d-833c-aa09db181193 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:18.974025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 98/194] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:18.976294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:19.013729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:19.014968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.015250 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.015528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] PATH_INFO: `/v3/users/2e58ecc0cf7640b59fba11a624e095d8` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.017030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=2e58ecc0cf7640b59fba11a624e095d8)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:19.019051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:19.021220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 97/195] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/users/2e58ecc0cf7640b59fba11a624e095d8 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:19.027679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:19.044096 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:19.111085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:19.125418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72119d2-1c13-4f7c-a4c1-dde44fe1f8d9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:19.157950 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535c3ea-0666-40ee-bcfc-bf07ed040eb7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:19.169722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:19.170815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.171123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.171601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.175099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:19.176698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:19.178754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 99/196] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:18 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:19.184903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:19.206197 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb22faa-610f-4167-94ac-9fec57c6cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:19.206936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22faa-610f-4167-94ac-9fec57c6cc5e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.207216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22faa-610f-4167-94ac-9fec57c6cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.207499 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb22faa-610f-4167-94ac-9fec57c6cc5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.299735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb22faa-610f-4167-94ac-9fec57c6cc5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:19.301466 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb22faa-610f-4167-94ac-9fec57c6cc5e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:19.302384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef18b94-eeda-4f7c-9442-1dcedcfa25cc None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:19.329813 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:19.360156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:19.361133 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.361602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.362862 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.363848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=2e58ecc0cf7640b59fba11a624e095d8) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:19.364821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:19.368529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:19.375238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 100/197] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:19 2022] GET /identity/v3/role_assignments?user.id=2e58ecc0cf7640b59fba11a624e095d8&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:19.384244 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:19.445014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 98/198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:34:19.447001 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:19.602064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb21ab-e19d-4153-878c-f3d8fcedf98b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.602423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb21ab-e19d-4153-878c-f3d8fcedf98b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.602644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb21ab-e19d-4153-878c-f3d8fcedf98b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.604182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 99/199] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:19.611614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.612231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.612709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.619128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e0ce-4eec-4b98-9175-9592a75d6152 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:19.633488 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:19.635079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:19.686054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:19.686945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.687218 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.687482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.688995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:19.690535 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:19.692326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 101/200] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:19 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:19.699726 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:19.747288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 100/201] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:19.770990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:19.797815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:19.895082 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143523-e35b-498b-9a0d-93f9153cf21c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:19.921903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:19.938658 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c02a1-16bb-4a7d-90e8-bca2d490bcf2 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:19.964269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:19.965428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:19.965957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:19.966438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:19.967514 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] RBAC: Adding query filter params (role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=2e58ecc0cf7640b59fba11a624e095d8) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:19.968054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:19.969790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:19.985080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 102/202] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:19 2022] GET /identity/v3/role_assignments?user.id=2e58ecc0cf7640b59fba11a624e095d8&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:20.015204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:20.016456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:20.016976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:20.017506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:20.034857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:20.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:20.051507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:20.052436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 101/203] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:20.059940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:20.192032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483003e9-ac4e-40e8-9918-4fe2c6f7be5e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:20.209658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:20.228562 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:20.229306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:20.229586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:20.229861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:20.230613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:20.230911 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:20.232315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:20.241843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 103/204] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:20.247705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:20.376615 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0709e28-6226-4e5f-a89e-8063d3caa17e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:20.411244 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:20.412182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:20.412459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:20.412725 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] PATH_INFO: `/v3/users/6e952459c8a44e96a841dc6915e0acdb` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:20.414016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] RBAC: Authorizing `identity:get_user(user_id=6e952459c8a44e96a841dc6915e0acdb)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:20.415500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:20.417145 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 102/205] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:20 2022] GET /identity/v3/users/6e952459c8a44e96a841dc6915e0acdb => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:20.422897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:20.536960 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9387c42-d782-47e8-b5a2-a00974894dc6 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:20.571458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:20.572118 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:20.572397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:20.572671 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:20.630298 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:20.641432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:20.643076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:20.745550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 104/206] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:20.751003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:20.772984 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:20.885369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae2c65-ede4-4ba2-aacd-ba2fd5ec418a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:20.926398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473371b-569e-4095-9e87-2f82e7981bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:20.927321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473371b-569e-4095-9e87-2f82e7981bef None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:20.927597 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473371b-569e-4095-9e87-2f82e7981bef None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:20.927866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473371b-569e-4095-9e87-2f82e7981bef None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:20.929096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473371b-569e-4095-9e87-2f82e7981bef None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:20.930641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473371b-569e-4095-9e87-2f82e7981bef None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:20.932708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 103/207] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:20 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:20.938839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:21.021640 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:21.042155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a99d55-aeae-4c46-9c2a-45f8d64454ef None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:21.076682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:21.077384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:21.077716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:21.077989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:21.078866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] RBAC: Adding query filter params (role.id=82a463123e20467bb4c64ad6789ecfc8, scope.project.id=266c22fce804450d859a7e09765df901, user.id=6e952459c8a44e96a841dc6915e0acdb) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:21.079176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:21.080633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211156c-2c6e-46cb-991e-4c78ca07c1a5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:21.087176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 105/208] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:20 2022] GET /identity/v3/role_assignments?user.id=6e952459c8a44e96a841dc6915e0acdb&scope.project.id=266c22fce804450d859a7e09765df901&role.id=82a463123e20467bb4c64ad6789ecfc8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:21.701165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923348a-2319-453d-aa1b-72844560fcd0 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:21.701528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923348a-2319-453d-aa1b-72844560fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:21.701748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923348a-2319-453d-aa1b-72844560fcd0 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:21.703340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 104/209] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:21.708809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:21.709306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:21.709407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:21.727051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:21.728299 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:21.808528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 106/210] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:21.834394 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923348a-2319-453d-aa1b-72844560fcd0 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:21.931421 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:21.947995 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923348a-2319-453d-aa1b-72844560fcd0 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:21.982253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af576de0-058f-4758-b0ba-089b3bde7c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:21.983067 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af576de0-058f-4758-b0ba-089b3bde7c74 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:21.983351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af576de0-058f-4758-b0ba-089b3bde7c74 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:21.983627 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af576de0-058f-4758-b0ba-089b3bde7c74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:21.994552 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af576de0-058f-4758-b0ba-089b3bde7c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-af576de0-058f-4758-b0ba-089b3bde7c74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:22.002540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:22.003875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:22.005209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 105/211] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:22.010184 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.050818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a4dce-8dd6-46c2-8381-75e548c35386 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.051144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a4dce-8dd6-46c2-8381-75e548c35386 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.051420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a4dce-8dd6-46c2-8381-75e548c35386 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.053651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 106/212] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:22.058976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9ed6-05b4-408c-bde6-81324fe8d69e None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.059468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9ed6-05b4-408c-bde6-81324fe8d69e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.060205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9ed6-05b4-408c-bde6-81324fe8d69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.082226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-e3db9ed6-05b4-408c-bde6-81324fe8d69e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:22.084634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-e3db9ed6-05b4-408c-bde6-81324fe8d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:22.089569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.124697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d386e0-b98f-4bfe-bbd7-ee7baecc8533 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.161002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.161218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 107/213] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:22.162057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.162390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.162670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.162842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964e13-b61a-4cbd-8858-696da9ca2b86 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.163139 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964e13-b61a-4cbd-8858-696da9ca2b86 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.163421 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964e13-b61a-4cbd-8858-696da9ca2b86 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.163531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:22.163791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.164713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 108/214] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:22.165219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.170455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.170731 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.171051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.179601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 107/215] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:22.182245 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.194335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:22.195179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:22.270705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 109/216] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:22.278437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.290297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98c87-75cc-4e5c-8c45-54b930effa69 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.290484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.326183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.332595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.333530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.333809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.334078 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] PATH_INFO: `/v3/users/152f7330854942ec93bb6ace7f73b0b6` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.349639 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] RBAC: Authorizing `identity:get_user(user_id=152f7330854942ec93bb6ace7f73b0b6)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.351420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.357731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 108/217] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/users/152f7330854942ec93bb6ace7f73b0b6 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:22.357731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.392056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404015dc-f6a8-401c-bfd0-446c6f61426e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.428463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.429649 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.429939 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.430214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.431478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.432941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.439143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 110/218] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:22.439143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.464499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaa4b1-b362-4e78-a1a5-9ac34360dc98 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.499436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.500271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.500554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.503438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.518401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:22.521563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:22.528542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:22.529968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:22.531013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:22.534991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 109/219] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:22.535882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.556941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3fe10-bf86-42fe-bda0-45b714b918f0 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.590099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.596135 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49101459-135e-4e56-8ca3-69a0982e2a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.596809 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49101459-135e-4e56-8ca3-69a0982e2a15 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.597089 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49101459-135e-4e56-8ca3-69a0982e2a15 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.597398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49101459-135e-4e56-8ca3-69a0982e2a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.646317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaeaf7-b70e-4963-949f-0ea532f6f379 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.677248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49101459-135e-4e56-8ca3-69a0982e2a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.679549 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49101459-135e-4e56-8ca3-69a0982e2a15 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.688915 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.689789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.690145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.690463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.691151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.692644 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.716773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 110/220] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:22 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:22.716773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.768970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.799025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 111/221] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:22.802840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc2dce-520e-4585-9593-53b8dd8cfc7a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.803131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc2dce-520e-4585-9593-53b8dd8cfc7a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.803411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc2dce-520e-4585-9593-53b8dd8cfc7a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.806040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 112/222] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:22.806040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc2dce-520e-4585-9593-53b8dd8cfc7a None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:22.839182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e07c6-8992-439b-b630-d4cdb3e06d7d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.842515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:22.885004 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.885998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.886431 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.887007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.887926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:22.888386 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.890005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07183912-7c4d-4e37-9c46-43bdd89b66c9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.900586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 111/223] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:22.901881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.902284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.902568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.916959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc2dce-520e-4585-9593-53b8dd8cfc7a None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:22.919888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:22.921070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:22.956126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:22.957002 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:22.960554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:22.960554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:22.960554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:22.960554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:22.963155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 113/224] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:22.963155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.000108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 112/225] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:23.000979 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.010067 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:23.078205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccf1fc-918c-42dd-8b79-2a6bb8a4593d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.115777 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.116767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.116978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.117302 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] PATH_INFO: `/v3/users/2e58ecc0cf7640b59fba11a624e095d8` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.118502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] RBAC: Authorizing `identity:get_user(user_id=2e58ecc0cf7640b59fba11a624e095d8)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:23.120129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:23.122185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 114/226] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:22 2022] GET /identity/v3/users/2e58ecc0cf7640b59fba11a624e095d8 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:23.122706 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.137080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:23.144883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d128-1910-4a59-bd15-b27f3d26c5b1 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.188405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.189697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.190187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.190807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.192147 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=152f7330854942ec93bb6ace7f73b0b6) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:23.192887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:23.195173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:23.202880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 113/227] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:23 2022] GET /identity/v3/role_assignments?user.id=152f7330854942ec93bb6ace7f73b0b6&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:23.204668 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.254650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e6482-7c74-444c-8d58-4847b6e8c763 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.284021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:23.341728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481812-2145-4798-af00-8979351fb807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.342588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481812-2145-4798-af00-8979351fb807 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.342864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481812-2145-4798-af00-8979351fb807 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.343292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481812-2145-4798-af00-8979351fb807 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.369031 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481812-2145-4798-af00-8979351fb807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9d481812-2145-4798-af00-8979351fb807 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:23.377474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:23.378612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:23.379521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 115/228] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:23.387261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481812-2145-4798-af00-8979351fb807 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.434035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5a1a6-37d1-4b60-9e3f-8f706aaad351 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.485584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.486928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.487447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.487931 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.489393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:23.492210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:23.494657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 114/229] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:23 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:23.500723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.555474 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:23.564177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481812-2145-4798-af00-8979351fb807 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.611971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.612682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.612968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.613281 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.614047 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:23.614354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:23.615792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:23.624995 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 116/230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:23.630587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.669763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:23.751333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14c566-19e9-4ab1-b516-3e56c4c006c9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.770621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a182414-79b5-429c-aad2-e63f0886aa67 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:23.819153 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.820134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.820376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.820709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/2e58ecc0cf7640b59fba11a624e095d8/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.823017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] RBAC: Authorizing `identity:create_grant(project_id=31dbad7c8bdb4192999277e9f519bc7b, user_id=2e58ecc0cf7640b59fba11a624e095d8, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:23.824618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:23.830975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:23.831877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:23.832820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:23.833205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] PATH_INFO: `/v3/users/2e58ecc0cf7640b59fba11a624e095d8` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:23.835024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] RBAC: Authorizing `identity:get_user(user_id=2e58ecc0cf7640b59fba11a624e095d8)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:23.840571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:23.843548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 115/231] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:23 2022] PUT /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/2e58ecc0cf7640b59fba11a624e095d8/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:23.845478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 117/232] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:23 2022] GET /identity/v3/users/2e58ecc0cf7640b59fba11a624e095d8 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:23.851376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:23.951753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:23.965275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f14329-2f32-4660-a36f-b91ebebbdeec None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:24.001690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6f2e3-a279-4108-8920-fffae5c3c2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:24.002581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6f2e3-a279-4108-8920-fffae5c3c2b9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:24.002860 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6f2e3-a279-4108-8920-fffae5c3c2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:24.003134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6f2e3-a279-4108-8920-fffae5c3c2b9 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:24.004397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6f2e3-a279-4108-8920-fffae5c3c2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:24.005942 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6f2e3-a279-4108-8920-fffae5c3c2b9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:24.008244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 116/233] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:23 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:24.013675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:24.120241 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:24.123673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365f2e6-47c2-4663-8a46-966b0ee0d99d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:24.165759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5810-eb0e-46b3-b249-5d7592231151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:24.166677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5810-eb0e-46b3-b249-5d7592231151 None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:24.167072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5810-eb0e-46b3-b249-5d7592231151 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:24.167206 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5810-eb0e-46b3-b249-5d7592231151 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/2e58ecc0cf7640b59fba11a624e095d8/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:24.171889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5810-eb0e-46b3-b249-5d7592231151 None admin] RBAC: Authorizing `identity:create_grant(project_id=31dbad7c8bdb4192999277e9f519bc7b, user_id=2e58ecc0cf7640b59fba11a624e095d8, role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:24.173405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5810-eb0e-46b3-b249-5d7592231151 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:24.195747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 118/234] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:24 2022] PUT /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/2e58ecc0cf7640b59fba11a624e095d8/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:34:25.253913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27991d-722d-4388-b75c-47fe52c3e90b None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.254483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27991d-722d-4388-b75c-47fe52c3e90b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.254483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27991d-722d-4388-b75c-47fe52c3e90b None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.256066 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 117/235] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:25.261668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca240ae-18a3-4192-a51f-4c6e12e9255e None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.262107 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca240ae-18a3-4192-a51f-4c6e12e9255e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.262202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca240ae-18a3-4192-a51f-4c6e12e9255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.286253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-dca240ae-18a3-4192-a51f-4c6e12e9255e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:25.287483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-dca240ae-18a3-4192-a51f-4c6e12e9255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:25.379982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 119/236] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:25.412630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27991d-722d-4388-b75c-47fe52c3e90b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:25.507211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386598ea-7312-455d-a155-c0413cd3e1af None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.507419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386598ea-7312-455d-a155-c0413cd3e1af None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.507705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386598ea-7312-455d-a155-c0413cd3e1af None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.509137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 120/237] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:25.523671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.524644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.525139 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.545951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:25.547151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:25.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27991d-722d-4388-b75c-47fe52c3e90b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:25.641083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:25.641968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.642249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.642519 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.650500 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:25.652062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:25.654384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 118/238] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:25.660806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:25.662379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 121/239] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:25.690428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:25.703554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:25.721709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:25.819388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86bad-a349-4fdd-aaa2-5536ee137668 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:25.835182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d90f0-86e1-468a-b955-98cd8c7ce809 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:25.868816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b17fa-03ff-46f0-ab9d-405a06946957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:25.869540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b17fa-03ff-46f0-ab9d-405a06946957 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.869820 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b17fa-03ff-46f0-ab9d-405a06946957 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.870066 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3029-878a-4812-982d-359967d83242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:25.870213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b17fa-03ff-46f0-ab9d-405a06946957 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.870848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b17fa-03ff-46f0-ab9d-405a06946957 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:25.871168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3029-878a-4812-982d-359967d83242 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:25.871541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3029-878a-4812-982d-359967d83242 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:25.871907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3029-878a-4812-982d-359967d83242 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:25.872323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b17fa-03ff-46f0-ab9d-405a06946957 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:25.882798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde3029-878a-4812-982d-359967d83242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-ebde3029-878a-4812-982d-359967d83242 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:25.895165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:25.896332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:25.897345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 122/240] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:25.905712 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3029-878a-4812-982d-359967d83242 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:25.913222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 119/241] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:25 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:34:26.002444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.011893 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb9262-8ac5-4803-be62-50f86d508553 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.012205 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb9262-8ac5-4803-be62-50f86d508553 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.012453 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb9262-8ac5-4803-be62-50f86d508553 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.015159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 120/242] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:26.022753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.023093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.023210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3029-878a-4812-982d-359967d83242 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.023316 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.061330 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:26.061575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.062043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.062322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.062593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.062766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:26.063359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:26.063664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:26.065262 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:26.075389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 123/243] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:26.081144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.190779 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f248e5d-167b-4f9e-8ea9-011556664f74 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.200067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 121/244] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:26.224053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.225674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.226797 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.227238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.227652 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] PATH_INFO: `/v3/users/1e3c4f5f245f4b279336e21f0050556f` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.237494 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.241703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.244841 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] RBAC: Authorizing `identity:get_user(user_id=1e3c4f5f245f4b279336e21f0050556f)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:26.247490 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:26.253560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 124/245] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/users/1e3c4f5f245f4b279336e21f0050556f => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:26.256104 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.349187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fca2a4-5963-4f66-b7e2-eac9dd36de87 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.380524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b7d3d-6e13-4fcc-8a6e-05d9736edd1e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.393909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.395017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.395558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.395983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.409603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:26.418710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:26.419784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:26.420725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:26.421447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.421447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.421447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.421447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.424012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 122/246] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:26.430076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.495635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.497112 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:26.498802 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:26.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.565988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729428-bf04-4ca7-ae95-b4d19960fa4c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.608611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.609751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.610276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.610762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.611886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:26.612401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:26.614232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:26.622210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 125/247] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:26.624506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 123/248] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:26.627907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.630715 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.750131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0625aa-8159-4e64-8cf8-4bc1750bca10 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.756034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.799169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15bdd0-62b1-4c3c-b758-570dd466e221 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:26.820430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.822066 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.822793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.823522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.826131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:26.828519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:26.831696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 126/249] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:26.839336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:26.862126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:26.873135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:26.875594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:26.876954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:26.877493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] PATH_INFO: `/v3/users/152f7330854942ec93bb6ace7f73b0b6` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:26.881975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=152f7330854942ec93bb6ace7f73b0b6)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:26.883869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:26.891505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 124/250] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/users/152f7330854942ec93bb6ace7f73b0b6 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:26.904050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.048494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd3f99-794b-4891-923d-f79a3c936e7a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.064396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fe2c9-19e5-4109-aa13-8c5130957ac2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.100542 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.101677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.102178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.102557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57682c7-2cc5-4758-9c93-d63e21dc3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.102924 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.103434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57682c7-2cc5-4758-9c93-d63e21dc3387 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.103710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57682c7-2cc5-4758-9c93-d63e21dc3387 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.103977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57682c7-2cc5-4758-9c93-d63e21dc3387 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.104622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] RBAC: Adding query filter params (role.id=82a463123e20467bb4c64ad6789ecfc8, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=1e3c4f5f245f4b279336e21f0050556f) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:27.105165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:27.105315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57682c7-2cc5-4758-9c93-d63e21dc3387 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:27.106664 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57682c7-2cc5-4758-9c93-d63e21dc3387 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:27.107179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e7450-2206-4a24-a3c5-d12804950825 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:27.108686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 125/251] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:27.109910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a468da-63f9-42a5-abee-70fa5ab76fa6 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.110203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a468da-63f9-42a5-abee-70fa5ab76fa6 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.110460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a468da-63f9-42a5-abee-70fa5ab76fa6 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.115774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 126/252] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:27.116939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a468da-63f9-42a5-abee-70fa5ab76fa6 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.120323 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 127/253] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:26 2022] GET /identity/v3/role_assignments?user.id=1e3c4f5f245f4b279336e21f0050556f&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=82a463123e20467bb4c64ad6789ecfc8 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:27.122770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f986c-0493-4ec8-92f5-da8afb0a15e7 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.123301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f986c-0493-4ec8-92f5-da8afb0a15e7 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.123800 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f986c-0493-4ec8-92f5-da8afb0a15e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.144485 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-051f986c-0493-4ec8-92f5-da8afb0a15e7 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:27.146382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-051f986c-0493-4ec8-92f5-da8afb0a15e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:27.180507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:27.234059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 128/254] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:27.236077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec341f73-6933-4117-aae9-47099539b10b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.236614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec341f73-6933-4117-aae9-47099539b10b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.237127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec341f73-6933-4117-aae9-47099539b10b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.239319 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 129/255] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:27.247584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.248326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.248879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.285222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:27.286463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:27.324612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:27.398102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a468da-63f9-42a5-abee-70fa5ab76fa6 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.414372 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 130/256] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:27.415961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.435290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.436197 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.436482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.436759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/152f7330854942ec93bb6ace7f73b0b6/roles/6b466334835d44468e51d8a0bfed2e86` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.446263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] RBAC: Authorizing `identity:create_grant(project_id=31dbad7c8bdb4192999277e9f519bc7b, user_id=152f7330854942ec93bb6ace7f73b0b6, role_id=6b466334835d44468e51d8a0bfed2e86)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:27.447779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:27.470445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 127/257] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:27 2022] PUT /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/152f7330854942ec93bb6ace7f73b0b6/roles/6b466334835d44468e51d8a0bfed2e86 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:27.471372 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.563406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:27.587477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3bd72-fb20-49e0-9509-6c8f302e3b27 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.625266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c19848-07d0-41c5-b09d-24621d6bf273 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.652309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:27.654760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.655990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.656494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.657034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.670921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.672280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.672821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.673357 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.680353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:27.694522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:27.695600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:27.696520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:27.702778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:27.705893 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 131/258] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:27.707852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:27.722559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:27.723661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:27.724589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:27.727292 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 128/259] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:27.733068 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.834852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8041-a79e-4bf6-8e5f-10dae82441b1 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.868354 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cd5a9-87fa-4b60-939f-7b8765cbd558 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:27.884164 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.885256 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.885777 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.886272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.887263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:27.887805 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:27.889491 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:27.908389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 132/260] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:27.913691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:27.941657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:27.944443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:27.945345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:27.945633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:27.945916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:27.946697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:27.947006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:27.950559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:27.971147 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 129/261] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:27.976890 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:28.044121 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:28.071732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bc08c-6f90-4607-850b-307002aca769 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:28.100227 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7b7e0-964e-4ba9-adf1-f429fdfda9e3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:28.124764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:28.125694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:28.125982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:28.126256 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] PATH_INFO: `/v3/users/2e58ecc0cf7640b59fba11a624e095d8` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:28.127486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] RBAC: Authorizing `identity:get_user(user_id=2e58ecc0cf7640b59fba11a624e095d8)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:28.128957 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:28.131034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 133/262] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:27 2022] GET /identity/v3/users/2e58ecc0cf7640b59fba11a624e095d8 => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:28.136321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:28.136517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:28.137368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:28.137653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:28.137928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] PATH_INFO: `/v3/users/2e58ecc0cf7640b59fba11a624e095d8` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:28.139542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] RBAC: Authorizing `identity:get_user(user_id=2e58ecc0cf7640b59fba11a624e095d8)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:28.141044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:28.143055 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 130/263] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:27 2022] GET /identity/v3/users/2e58ecc0cf7640b59fba11a624e095d8 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:28.148906 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:28.308971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:28.331296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14fadd-04ba-4ce6-b8be-1a3e63c23998 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:28.339365 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a094f-b916-4f6b-b162-d17a815c0497 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:28.376528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:28.381797 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:28.382477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:28.382756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:28.383029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:28.407606 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:28.409779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:28.410237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:28.410520 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:28.456291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:28.458665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:28.553270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:28.555340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:28.576137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:28.607147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 134/264] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:28.608600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:28.737753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 131/265] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:28.746249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:28.782534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:28.795594 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6162747-d9d8-4fa2-97f2-c3010f54aa41 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:28.862530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:28.863436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:28.863711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:28.863982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:28.867563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:28.869119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:28.871100 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 135/266] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:28 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:28.876691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:28.911267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f4118-3fb0-4690-a863-e893150a44e0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:28.985868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:28.986779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:28.987054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:28.987332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:28.992653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:28.994182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:28.995964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 132/267] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:28 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:29.002020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:29.006734 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:29.103887 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037688c4-c474-4dda-b273-e3a90808d87c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:29.169713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:29.170741 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.170741 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.170961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.171804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=2e58ecc0cf7640b59fba11a624e095d8) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:29.172008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:29.173610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fc8a9-43f8-40da-93c3-730e29b71763 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:29.185539 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 136/268] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:28 2022] GET /identity/v3/role_assignments?user.id=2e58ecc0cf7640b59fba11a624e095d8&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:29.222851 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:29.249802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb171a2-085b-4bca-8428-371d59a485da None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:29.311058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:29.311741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.312021 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.312294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.313147 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] RBAC: Adding query filter params (role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=2e58ecc0cf7640b59fba11a624e095d8) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:29.313473 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:29.314983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e7a2-051d-40d0-ae1e-664ac5ddeec3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:29.326811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 133/269] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:29 2022] GET /identity/v3/role_assignments?user.id=2e58ecc0cf7640b59fba11a624e095d8&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:29.641587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f76e7-0f04-4038-bc9d-30d56b809c69 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.642137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f76e7-0f04-4038-bc9d-30d56b809c69 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.642137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f76e7-0f04-4038-bc9d-30d56b809c69 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.643794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 137/270] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:29.649062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.649912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.650013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.669838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:29.671046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:29.689022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8fe0-93c1-4f80-b569-e9324aacdaf9 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.689377 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8fe0-93c1-4f80-b569-e9324aacdaf9 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.689617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8fe0-93c1-4f80-b569-e9324aacdaf9 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.691275 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 138/271] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:29.696691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7c87-118c-4029-af42-24d73379e791 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.696980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7c87-118c-4029-af42-24d73379e791 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7c87-118c-4029-af42-24d73379e791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.722277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-829e7c87-118c-4029-af42-24d73379e791 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:29.723499 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-829e7c87-118c-4029-af42-24d73379e791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:29.749221 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 134/272] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:29.799094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:29.816568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 139/273] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:29.844182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e7c87-118c-4029-af42-24d73379e791 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:29.896412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:29.897187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:29.919153 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f4afe-49c6-4ada-be8c-7e0aa0c2cab4 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:29.957105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:29.958036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.958374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.958649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:29.959213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e7c87-118c-4029-af42-24d73379e791 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:29.959865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:29.961500 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:29.963603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 135/274] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:29.968892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:29.996454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:29.997673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:29.998187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:29.998713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.009763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:30.017929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:30.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:30.019883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:30.022447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 140/275] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:30.029281 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:30.077974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66c293-df63-42f2-8166-eb5e49c60312 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:30.117946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:30.118650 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.118993 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.119370 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.120260 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:30.121844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:30.142360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61652191-4d04-4160-bcb6-cc3efb734386 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:30.146089 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:30.155567 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 136/276] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:29 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:34:30.191552 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:30.192255 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.192606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.192946 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.193927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:30.194322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:30.195799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110c8a-ba92-42fb-825f-e018c025aed9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:30.206591 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 141/277] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:30.212831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:30.217050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:30.311211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1cfd5-c7e2-4a57-9f05-24a02a5981a2 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.311918 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1cfd5-c7e2-4a57-9f05-24a02a5981a2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.312422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1cfd5-c7e2-4a57-9f05-24a02a5981a2 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.314437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 142/278] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:30.320383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.321021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.321572 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.336957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf258-97f8-4dc4-ab8e-db49c480aa2f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:30.340451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:30.342005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:30.380827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:30.382257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.382764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.383069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] PATH_INFO: `/v3/users/82bbb40717e54348a4114bc59cae5c53` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.398076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] RBAC: Authorizing `identity:get_user(user_id=82bbb40717e54348a4114bc59cae5c53)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:30.400012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:30.402399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 137/279] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/users/82bbb40717e54348a4114bc59cae5c53 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:30.407595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:30.425824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 143/280] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:30.448048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:30.476766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:30.537672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658be94-ec30-4c25-95e1-7eebe45ec2b3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:30.571220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:30.583231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e44ce8-54ca-4ac1-9d4a-c8473462b434 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:30.587587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:30.588551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.588999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.589489 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.590507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, user.id=82bbb40717e54348a4114bc59cae5c53, scope.system=all) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:30.590983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:30.592663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:30.600779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 138/281] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/role_assignments?user.id=82bbb40717e54348a4114bc59cae5c53&scope.system=all&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:30.624967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefdeb2-8f4c-4c67-ae38-0a459e10bb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:30.626145 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefdeb2-8f4c-4c67-ae38-0a459e10bb3e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.626578 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefdeb2-8f4c-4c67-ae38-0a459e10bb3e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.627017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefdeb2-8f4c-4c67-ae38-0a459e10bb3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.640832 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefdeb2-8f4c-4c67-ae38-0a459e10bb3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-1aefdeb2-8f4c-4c67-ae38-0a459e10bb3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:30.649723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:30.651266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:30.652278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 144/282] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:30.657010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:30.780692 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d8843-f928-43b4-9c61-4e34d7078db0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:30.787103 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dfd92-84b3-4fa8-9c95-f68db3c60fae None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.787430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dfd92-84b3-4fa8-9c95-f68db3c60fae None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.787730 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dfd92-84b3-4fa8-9c95-f68db3c60fae None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.790908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 145/283] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:30.795618 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.795941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.796164 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.814552 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:30.815784 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:30.822455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:30.850510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:30.851220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:30.851500 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:30.851772 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:30.852534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:30.852838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:30.854278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:30.876645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 139/284] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:30.881926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:30.928387 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 146/285] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:30.983191 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:31.054401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dcc5c-4e82-43c6-a2ae-572a3c5cbc62 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.117717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:31.122247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310fe5d-78b1-4e70-a27f-0fd29cf99685 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.137271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.137271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.137271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.137271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] PATH_INFO: `/v3/users/1e3c4f5f245f4b279336e21f0050556f` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:31.137722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] RBAC: Authorizing `identity:get_user(user_id=1e3c4f5f245f4b279336e21f0050556f)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:31.138595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:31.140780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 140/286] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/users/1e3c4f5f245f4b279336e21f0050556f => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:31.148231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:31.172015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.173161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.174368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.175031 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:31.198315 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:31.212675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:31.213787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:31.214714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:31.221801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 147/287] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:31.228497 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:31.249265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:31.287735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39393f20-1684-4348-8651-ce8a7cd68173 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.327319 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.329146 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.329449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.329728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:31.331700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:31.333196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:31.341707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 141/288] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:31 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:31.345995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:31.439699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78325b9-a160-4eda-a4ac-9934c029b742 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.508486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.509190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.509483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.509756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:31.512712 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:31.512712 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:31.512712 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:31.534016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 148/289] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:31.538312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:31.539172 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41b468-7f8a-4e8c-8a08-eb504225186f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.565611 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:31.586144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:31.606801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.607774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.608055 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.608334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/1e3c4f5f245f4b279336e21f0050556f/roles/82a463123e20467bb4c64ad6789ecfc8` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:31.615097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=31dbad7c8bdb4192999277e9f519bc7b, user_id=1e3c4f5f245f4b279336e21f0050556f, role_id=82a463123e20467bb4c64ad6789ecfc8)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:31.616658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:31.639358 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 142/290] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:34:31 2022] PUT /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/users/1e3c4f5f245f4b279336e21f0050556f/roles/82a463123e20467bb4c64ad6789ecfc8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:34:31.711702 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed0fd2-6481-49ef-85a5-840a1424f8b3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.752838 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.753799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.754081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.754360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] PATH_INFO: `/v3/users/152f7330854942ec93bb6ace7f73b0b6` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:31.755659 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] RBAC: Authorizing `identity:get_user(user_id=152f7330854942ec93bb6ace7f73b0b6)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:31.757264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:31.759275 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 149/291] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:31 2022] GET /identity/v3/users/152f7330854942ec93bb6ace7f73b0b6 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:31.766025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:31.896866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0718-6b54-41a6-ae12-48a93a1691b9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:31.943722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:31.944827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:31.945397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:31.945898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:32.020317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:32.042222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:32.044057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:32.183230 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 143/292] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:32.188651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:32.299602 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:32.322799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81d27f-447e-449f-b458-94e6d2d24f08 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:32.367504 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a88cf-ca67-484d-bc2f-190b4dc5b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:32.368389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a88cf-ca67-484d-bc2f-190b4dc5b529 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:32.368672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a88cf-ca67-484d-bc2f-190b4dc5b529 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:32.368948 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a88cf-ca67-484d-bc2f-190b4dc5b529 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:32.370182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a88cf-ca67-484d-bc2f-190b4dc5b529 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:32.371644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a88cf-ca67-484d-bc2f-190b4dc5b529 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:32.373707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 150/293] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:32 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:32.381276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:32.510085 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:32.528510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18c38-dbf9-4d1b-a362-a83a790d7202 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:32.576793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:32.577417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:32.577697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:32.577967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:32.578809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=152f7330854942ec93bb6ace7f73b0b6) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:32.579114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:32.580575 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44f083-df9d-4d1e-bd02-a9e07e0532b7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:32.587601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 144/294] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:32 2022] GET /identity/v3/role_assignments?user.id=152f7330854942ec93bb6ace7f73b0b6&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:32.992248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358167-c418-451f-b247-8ab5aef2ebfc None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:32.993143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358167-c418-451f-b247-8ab5aef2ebfc None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:32.993707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358167-c418-451f-b247-8ab5aef2ebfc None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:32.995590 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 151/295] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:33.000696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dbf30-3999-44ec-a2cd-64642d08fa25 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.001649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dbf30-3999-44ec-a2cd-64642d08fa25 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.002156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dbf30-3999-44ec-a2cd-64642d08fa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.021025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-061dbf30-3999-44ec-a2cd-64642d08fa25 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:33.022376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-061dbf30-3999-44ec-a2cd-64642d08fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:33.107833 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 145/296] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:33.134255 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358167-c418-451f-b247-8ab5aef2ebfc None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.254035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542b4e2-2fad-44b2-88e7-7b572ca558a2 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.254254 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542b4e2-2fad-44b2-88e7-7b572ca558a2 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.254505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542b4e2-2fad-44b2-88e7-7b572ca558a2 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.255906 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 146/297] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:33.261444 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba48ce-c336-4cde-9c93-8fd2f7ac3668 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.261990 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba48ce-c336-4cde-9c93-8fd2f7ac3668 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.262581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba48ce-c336-4cde-9c93-8fd2f7ac3668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.263934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358167-c418-451f-b247-8ab5aef2ebfc None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:33.280920 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-7aba48ce-c336-4cde-9c93-8fd2f7ac3668 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:33.282502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-7aba48ce-c336-4cde-9c93-8fd2f7ac3668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:33.301204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82295dd-6613-4698-8a50-dcc7143c2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:33.302086 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82295dd-6613-4698-8a50-dcc7143c2477 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.302368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82295dd-6613-4698-8a50-dcc7143c2477 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.302703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82295dd-6613-4698-8a50-dcc7143c2477 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.303961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82295dd-6613-4698-8a50-dcc7143c2477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:33.305470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82295dd-6613-4698-8a50-dcc7143c2477 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:33.307236 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 152/298] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:33.308505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb376-8a76-4db5-8efd-b10c5f9cbe98 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.308788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb376-8a76-4db5-8efd-b10c5f9cbe98 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.309062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb376-8a76-4db5-8efd-b10c5f9cbe98 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.310315 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 153/299] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:33.314476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bb376-8a76-4db5-8efd-b10c5f9cbe98 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.330745 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:33.354646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 147/300] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:33.361407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.361407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.361407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.381964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:33.383866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:33.386759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:33.427426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bb376-8a76-4db5-8efd-b10c5f9cbe98 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:33.464205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:33.465202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.465643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.465914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] PATH_INFO: `/v3/groups` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.466605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:33.468053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:33.468922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 148/301] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:33.473665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.497588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 154/302] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 14:34:33 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:33.498838 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.613395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26000505-23d4-47bb-9221-41cb5574bd8b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:33.615991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:33.652996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0e405-ba81-40ba-a8d6-60a8dddb216a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:33.663095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:33.664549 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.664942 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.665351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.675843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:33.682682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:33.694139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:33.695408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:33.696538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:33.697658 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:33.698349 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 149/303] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:33.698841 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.699288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.699695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.704296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.714642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:33.723729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:33.724803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:33.725969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 155/304] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:33.730747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.846558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252c96d-58be-4d7a-97e8-1b86da3653c7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:33.849500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fea37-d6a1-40ee-9f27-853ca40e87e5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:33.880680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:33.886836 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:33.887818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.888322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.888902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.890300 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:33.890845 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:33.892486 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:33.893017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:33.893373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:33.893657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:33.893928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:33.894684 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:33.894988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:33.896669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:33.904767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 156/305] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:33.907270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 150/306] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:33.910452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.912767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:33.995462 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:34.046884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85dbd9-98ab-427a-8636-68b596ecf268 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.047555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac599a-5d56-4550-a914-0cfc0f6dccdc None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.090056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.091017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.091204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.091323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.091557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] PATH_INFO: `/v3/users/82bbb40717e54348a4114bc59cae5c53` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.092133 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.092328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.092601 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] PATH_INFO: `/v3/users/8191d30b64f64a5389d1c955b9769553` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.092782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=82bbb40717e54348a4114bc59cae5c53)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:34.094491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:34.101389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 151/307] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/users/82bbb40717e54348a4114bc59cae5c53 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:34.103673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:34.109775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] RBAC: Authorizing `identity:get_user(user_id=8191d30b64f64a5389d1c955b9769553)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:34.111232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:34.118010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 157/308] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:33 2022] GET /identity/v3/users/8191d30b64f64a5389d1c955b9769553 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:34.118859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:34.176138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:34.231601 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40925060-0017-4f61-94fa-1bb37a8de7af None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.247749 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62fcb9-2aaf-4fa0-9822-4c6fa7e5f0f9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.267334 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.268127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.268505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.268954 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.269853 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] RBAC: Adding query filter params (role.id=82a463123e20467bb4c64ad6789ecfc8, user.id=8191d30b64f64a5389d1c955b9769553, scope.system=all) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:34.270260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:34.271799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f58be-e76e-4b78-93c7-de30af9d9efd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:34.281668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 158/309] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:34:34 2022] GET /identity/v3/role_assignments?user.id=8191d30b64f64a5389d1c955b9769553&scope.system=all&role.id=82a463123e20467bb4c64ad6789ecfc8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:34.285661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:34.293510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69cd0b-3fb2-49bc-82a8-2c3e6ff2e361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.294695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69cd0b-3fb2-49bc-82a8-2c3e6ff2e361 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.295178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69cd0b-3fb2-49bc-82a8-2c3e6ff2e361 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.295617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69cd0b-3fb2-49bc-82a8-2c3e6ff2e361 None admin] PATH_INFO: `/v3/system/users/82bbb40717e54348a4114bc59cae5c53/roles/6b466334835d44468e51d8a0bfed2e86` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.297483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69cd0b-3fb2-49bc-82a8-2c3e6ff2e361 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=82bbb40717e54348a4114bc59cae5c53, role_id=6b466334835d44468e51d8a0bfed2e86)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:34.299149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69cd0b-3fb2-49bc-82a8-2c3e6ff2e361 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:34.317672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 152/310] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 14:34:34 2022] PUT /identity/v3/system/users/82bbb40717e54348a4114bc59cae5c53/roles/6b466334835d44468e51d8a0bfed2e86 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:34.362225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea0ed-efbc-489d-9c21-f846882f0f06 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.362774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea0ed-efbc-489d-9c21-f846882f0f06 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.363149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea0ed-efbc-489d-9c21-f846882f0f06 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.365021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 159/311] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:34.370968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.371301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.371522 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.391292 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:34.392591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:34.480835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 153/312] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:34.508626 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea0ed-efbc-489d-9c21-f846882f0f06 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:34.618220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea0ed-efbc-489d-9c21-f846882f0f06 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.635402 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:34.654988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.655859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.656138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.656411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.671708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:34.679867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:34.680928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:34.681914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 160/313] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:34.687197 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:34.793830 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b614d-735f-4ce1-8358-44d1a98aa203 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.818146 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:34.831149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.832165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.832448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.832725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.833656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:34.833991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:34.835560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:34.845531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 154/314] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:34.851003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:34.952562 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b180-08ea-4aee-ae06-f972b9c740ae None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:34.987966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:34.988870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:34.989150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:34.989453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] PATH_INFO: `/v3/users/1e3c4f5f245f4b279336e21f0050556f` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:34.990722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] RBAC: Authorizing `identity:get_user(user_id=1e3c4f5f245f4b279336e21f0050556f)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:34.992198 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:34.994203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 161/315] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:34 2022] GET /identity/v3/users/1e3c4f5f245f4b279336e21f0050556f => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:35.002364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:35.107652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749009f-ab6c-4962-bd5a-c89586401f61 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:35.142219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:35.142883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:35.143166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:35.143445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:35.198665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:35.215532 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:35.216967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:35.320095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 155/316] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:35.325972 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:35.374289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:35.430825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2b42-c2ef-4102-8508-adc21d55039b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:35.464790 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ccf92-2b40-420f-aac7-371ab1817be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:35.465761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ccf92-2b40-420f-aac7-371ab1817be8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:35.465978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ccf92-2b40-420f-aac7-371ab1817be8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:35.466309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ccf92-2b40-420f-aac7-371ab1817be8 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:35.467532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ccf92-2b40-420f-aac7-371ab1817be8 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:35.468993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ccf92-2b40-420f-aac7-371ab1817be8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:35.471055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 162/317] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:35 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:35.477397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:35.566086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:35.582796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cf443-d779-4dc9-8a52-69b618e05add None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:35.618891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:35.619563 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:35.619841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:35.620108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:35.620941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] RBAC: Adding query filter params (role.id=82a463123e20467bb4c64ad6789ecfc8, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b, user.id=1e3c4f5f245f4b279336e21f0050556f) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:35.621276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:35.622726 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27194e0f-6a91-4aa5-b69b-00b09cf3a04b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:35.629337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 156/318] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:34:35 2022] GET /identity/v3/role_assignments?user.id=1e3c4f5f245f4b279336e21f0050556f&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=82a463123e20467bb4c64ad6789ecfc8 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:35.903856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbabb0-31c5-4167-a242-04fba1dcd885 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:35.904178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbabb0-31c5-4167-a242-04fba1dcd885 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:35.904640 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbabb0-31c5-4167-a242-04fba1dcd885 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:35.906304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 163/319] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:35.911758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:35.912053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:35.912343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:35.930004 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:35.931190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:36.000955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 157/320] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:36.023824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbabb0-31c5-4167-a242-04fba1dcd885 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:36.135681 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbabb0-31c5-4167-a242-04fba1dcd885 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:36.150787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:36.174146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6edb53-acb5-4767-9725-34cc7358ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:36.174911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6edb53-acb5-4767-9725-34cc7358ab29 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.175196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6edb53-acb5-4767-9725-34cc7358ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.175477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6edb53-acb5-4767-9725-34cc7358ab29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.176668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6edb53-acb5-4767-9725-34cc7358ab29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:36.178222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6edb53-acb5-4767-9725-34cc7358ab29 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:36.180088 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 164/321] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:36.185867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:36.280455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:36.305145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321c303-5206-42db-8b95-feb6ad419a69 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:36.341051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7089f-43c8-4048-8f42-3923aa39e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:36.341771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7089f-43c8-4048-8f42-3923aa39e4fc None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.342039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7089f-43c8-4048-8f42-3923aa39e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.342307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7089f-43c8-4048-8f42-3923aa39e4fc None admin] PATH_INFO: `/v3/groups` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.343008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7089f-43c8-4048-8f42-3923aa39e4fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:36.344539 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7089f-43c8-4048-8f42-3923aa39e4fc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:36.369267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 158/322] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 14:34:36 2022] POST /identity/v3/groups => generated 226 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:34:36.629785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c182ff0-7313-4ecc-bed5-ffd0e97b98ff None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.630370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c182ff0-7313-4ecc-bed5-ffd0e97b98ff None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.630370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c182ff0-7313-4ecc-bed5-ffd0e97b98ff None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.631953 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 165/323] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:36.637332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.637626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.637907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.656032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:36.657385 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:36.715567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ca86-f774-4f55-9b5b-7ed1988b1b5b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.715904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ca86-f774-4f55-9b5b-7ed1988b1b5b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.716127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ca86-f774-4f55-9b5b-7ed1988b1b5b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.717552 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 166/324] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:36.723433 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.723751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.724273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.731225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 159/325] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:36.744967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:36.746205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:36.753991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:36.830341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 167/326] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:36.843524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:36.859370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:36.886001 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87b28e-5902-4075-9b47-9a72538ad4de None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:36.931720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:36.932895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:36.933269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:36.933621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:36.953962 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:36.963648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:36.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:36.965781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 160/327] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:36.972315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:36.992999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242680-a5d0-432d-b19e-3ef10e4e9061 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.011883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:37.053952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.055288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.055288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.055751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.071136 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:37.082219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:37.083463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:37.084610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:37.085717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 168/328] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:37.094023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:37.110511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638d8c5-c881-42bd-a9e0-839720150ee6 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.141856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:37.157501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.158201 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.158537 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.158812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.159639 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:37.160141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:37.161373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:37.171529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 161/329] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:37.176555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:37.242810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288ce3d-59be-4fe0-bc92-e187e4eb8945 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.293408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.294210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.297458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.297838 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.298749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:37.299128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:37.300630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:37.310894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 169/330] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:37.313596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034f56e-2b01-4844-bd53-ce4e410a9ed3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.322300 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:37.354003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.355577 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.356067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.356767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] PATH_INFO: `/v3/users/8191d30b64f64a5389d1c955b9769553` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.358596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] RBAC: Authorizing `identity:get_user(user_id=8191d30b64f64a5389d1c955b9769553)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:37.360255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:37.363164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 162/331] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:37 2022] GET /identity/v3/users/8191d30b64f64a5389d1c955b9769553 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:37.368523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:37.383740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:37.479232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d906-ead0-40de-9187-f666fe882c12 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.521083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.522789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.523251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.523709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] PATH_INFO: `/v3/users/82bbb40717e54348a4114bc59cae5c53` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.525506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] RBAC: Authorizing `identity:get_user(user_id=82bbb40717e54348a4114bc59cae5c53)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:37.527750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:37.529974 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 170/332] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:37 2022] GET /identity/v3/users/82bbb40717e54348a4114bc59cae5c53 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:37.538391 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:37.546189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:37.620638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1ea8b-f1c4-409c-a595-5ba99dd042ea None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.672646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1976b-a595-49b0-a146-a90a1cecfa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.673706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1976b-a595-49b0-a146-a90a1cecfa5c None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.673992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1976b-a595-49b0-a146-a90a1cecfa5c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.674360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1976b-a595-49b0-a146-a90a1cecfa5c None admin] PATH_INFO: `/v3/system/users/8191d30b64f64a5389d1c955b9769553/roles/82a463123e20467bb4c64ad6789ecfc8` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.676257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1976b-a595-49b0-a146-a90a1cecfa5c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8191d30b64f64a5389d1c955b9769553, role_id=82a463123e20467bb4c64ad6789ecfc8)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:37.677903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1976b-a595-49b0-a146-a90a1cecfa5c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:37.690082 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 163/333] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 14:34:37 2022] PUT /identity/v3/system/users/8191d30b64f64a5389d1c955b9769553/roles/82a463123e20467bb4c64ad6789ecfc8 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:34:37.728557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:37.732794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fabbe-7b8b-4872-b292-f1244629f392 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:37.774637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:37.775312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:37.775592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:37.775867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:37.776728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] RBAC: Adding query filter params (role.id=6b466334835d44468e51d8a0bfed2e86, user.id=82bbb40717e54348a4114bc59cae5c53, scope.system=all) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:37.777040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:37.778540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3c1e5-8d39-4b76-b22f-1988438f9a1d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:37.795834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 171/334] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:34:37 2022] GET /identity/v3/role_assignments?user.id=82bbb40717e54348a4114bc59cae5c53&scope.system=all&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 510 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:39.207293 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169d3d6-4dc2-4757-9fb2-99a038b22cb9 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:39.207835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169d3d6-4dc2-4757-9fb2-99a038b22cb9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:39.207835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169d3d6-4dc2-4757-9fb2-99a038b22cb9 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:39.209795 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 164/335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:39.216263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:39.217158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:39.217664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:39.237276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:39.238896 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:39.327203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 172/336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:39.372138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169d3d6-4dc2-4757-9fb2-99a038b22cb9 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:39.485009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169d3d6-4dc2-4757-9fb2-99a038b22cb9 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:39.524521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:39.542103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:39.542918 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:39.543205 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:39.543466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:39.562187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:39.571258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:39.572899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:39.573936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 165/337] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:39.578581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:39.684331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:39.703270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae26fe-bf45-422c-bb11-b6b8334fd1d4 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:39.748403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:39.749029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:39.749348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:39.749623 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:39.750388 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:39.750692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:39.752107 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:39.761877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 173/338] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:39.767623 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:39.882868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f96f4-ca29-419b-b807-79169c1dfb12 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:39.925548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94226067-17a6-451c-860f-3d8b9e28402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:39.926227 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94226067-17a6-451c-860f-3d8b9e28402e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:39.926507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94226067-17a6-451c-860f-3d8b9e28402e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:39.926779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94226067-17a6-451c-860f-3d8b9e28402e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:39.996989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:40.019128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94226067-17a6-451c-860f-3d8b9e28402e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:40.020582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94226067-17a6-451c-860f-3d8b9e28402e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:40.141835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 166/339] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:40.147756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:40.255547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b93c5-4692-4087-9f0d-1841d4418258 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.256133 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b93c5-4692-4087-9f0d-1841d4418258 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.256133 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b93c5-4692-4087-9f0d-1841d4418258 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.261296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 167/340] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:40.270569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dc59e-85f6-4a6a-884a-42b5032b8e09 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.270763 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dc59e-85f6-4a6a-884a-42b5032b8e09 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.271031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dc59e-85f6-4a6a-884a-42b5032b8e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.274019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c012e41-9cf4-4fd7-ad56-5be34a6d5367 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:40.310611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-c89dc59e-85f6-4a6a-884a-42b5032b8e09 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:40.311867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-c89dc59e-85f6-4a6a-884a-42b5032b8e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:40.315052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:40.316971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.317359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.317648 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.319124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:40.320621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:40.322773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 174/341] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:40 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:40.328448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:40.360377 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:40.465196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 168/342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:40.467155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c5fb2-79a0-47cf-a417-d2778fea803e None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.468189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c5fb2-79a0-47cf-a417-d2778fea803e None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.468708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c5fb2-79a0-47cf-a417-d2778fea803e None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.470881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 169/343] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:40.476481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f0b21-7e53-4e4b-83bd-97fc25ed7a86 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.477035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f0b21-7e53-4e4b-83bd-97fc25ed7a86 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.477163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e131b27-81fe-4378-968f-5248484adf69 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:40.477721 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f0b21-7e53-4e4b-83bd-97fc25ed7a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.507846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-033f0b21-7e53-4e4b-83bd-97fc25ed7a86 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:40.515092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-033f0b21-7e53-4e4b-83bd-97fc25ed7a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:40.526480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:40.561965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:40.562863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.563144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.563414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.574306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:40.576199 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:40.578792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 175/344] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:40 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:40.579986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:40.599613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 170/345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:40.601629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54625998-92b2-4919-b6da-54e7ec1101b0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.602238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54625998-92b2-4919-b6da-54e7ec1101b0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.602796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54625998-92b2-4919-b6da-54e7ec1101b0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.604293 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 171/346] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:40.609723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54625998-92b2-4919-b6da-54e7ec1101b0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:40.679354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:40.700778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b66e14-7945-41ce-8a53-7ae6f466a2b2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:40.726148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54625998-92b2-4919-b6da-54e7ec1101b0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:40.736502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b001bcb-08b0-448c-a10b-b58c6b08bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:40.737609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b001bcb-08b0-448c-a10b-b58c6b08bac7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.737609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b001bcb-08b0-448c-a10b-b58c6b08bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.737907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b001bcb-08b0-448c-a10b-b58c6b08bac7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.751143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b001bcb-08b0-448c-a10b-b58c6b08bac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-4b001bcb-08b0-448c-a10b-b58c6b08bac7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:40.760614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:40.761669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:40.763019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 176/347] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:40.763795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.763795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.763951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.769289 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:40.770332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.770840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.771331 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.772631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=266c22fce804450d859a7e09765df901) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:40.773292 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:40.775454 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:40.782100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 172/348] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:40 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=266c22fce804450d859a7e09765df901&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:40.783294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:40.788258 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:40.789573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:40.793795 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:40.886355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 177/349] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:40.886355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:40.908967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e88a49-5c31-49e4-a391-f493ece31651 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:40.964015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:40.965452 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:40.966214 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:40.967460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:40.992939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:41.002428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:41.004780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:41.005872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:41.006916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:41.009969 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 173/350] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:41.011776 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.087517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adad78-31a1-4166-a266-4f7ab4d85767 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.096302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.138171 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f711-0113-4c79-9276-7d8a26635299 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.144038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.144998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.145540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.146041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.147185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:41.147717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:41.149811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:41.161192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 178/351] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:41.162777 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.190524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.191832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.192115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.192388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.205725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:41.215959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:41.217037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:41.218081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:41.221807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 174/352] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:41.222670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.344935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ad8c8-6203-45e3-9c0e-f3992c3a2d25 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.375827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.383388 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41fd8-eee6-4abb-9db8-dbb7634f4aba None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.385911 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.386591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.386877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.387154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.464189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:41.465803 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:41.469984 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.470690 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.470967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.471235 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.471982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:41.472282 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:41.473716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:41.475496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.489719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 179/353] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:41.490781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.576315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 175/354] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:41.581484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.614333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.658900 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6d6d4-e55d-4ef8-b151-ed1c5b36ca28 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.694715 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ed02-85ec-4149-bd62-783c38ba6f4a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.718987 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.719684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.719963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.720241 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.720983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:41.721292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:41.722672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:41.734379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18139c-4cfb-44a8-95bd-c1e1307d5db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.735048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18139c-4cfb-44a8-95bd-c1e1307d5db5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.735327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18139c-4cfb-44a8-95bd-c1e1307d5db5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.735599 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18139c-4cfb-44a8-95bd-c1e1307d5db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.735940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 180/355] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:41.736813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.812573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18139c-4cfb-44a8-95bd-c1e1307d5db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:41.814466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18139c-4cfb-44a8-95bd-c1e1307d5db5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:41.828972 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.849986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:41.889382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b30ee-71dc-473f-b619-7f1e7146ef6e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:41.935167 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:41.937175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.937495 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.937770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.939116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:41.940604 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:41.942701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 181/356] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:41.943291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:41.950517 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 176/357] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:41.951935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299609c-ca81-4f2c-b64b-dda977c73a13 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:41.952232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299609c-ca81-4f2c-b64b-dda977c73a13 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:41.952510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299609c-ca81-4f2c-b64b-dda977c73a13 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:41.954691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 177/358] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:41.955781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299609c-ca81-4f2c-b64b-dda977c73a13 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.061752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e219c7-faa4-4108-8cb7-542ee09ee265 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.088527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299609c-ca81-4f2c-b64b-dda977c73a13 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.092708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:42.104068 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.104745 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.105023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.105360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.123781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:42.134953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.135833 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.136140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.136444 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.137691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.139197 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.141035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 178/359] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:42.143028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.200334 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.203554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.278332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e7a20-70a0-4188-b91f-46ba13d4cd6a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.322866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64eaff-6cf1-46a6-aad2-bc9fc7477b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.324808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64eaff-6cf1-46a6-aad2-bc9fc7477b78 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.325209 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64eaff-6cf1-46a6-aad2-bc9fc7477b78 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.325598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64eaff-6cf1-46a6-aad2-bc9fc7477b78 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.326886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64eaff-6cf1-46a6-aad2-bc9fc7477b78 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.328352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64eaff-6cf1-46a6-aad2-bc9fc7477b78 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.330948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 179/360] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:42.332167 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276145-b124-4f12-9017-6cc134106023 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.332497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276145-b124-4f12-9017-6cc134106023 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.332781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276145-b124-4f12-9017-6cc134106023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.350098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 182/361] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:42.351328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.354804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-20276145-b124-4f12-9017-6cc134106023 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:42.356007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-20276145-b124-4f12-9017-6cc134106023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:42.388837 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:42.427781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:42.449265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 180/362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:42.449265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f53d31-de0c-402c-af56-28a46dbbbfd6 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.449265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f53d31-de0c-402c-af56-28a46dbbbfd6 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.449265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f53d31-de0c-402c-af56-28a46dbbbfd6 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.449265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 181/363] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:42.449265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f53d31-de0c-402c-af56-28a46dbbbfd6 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.497715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bc673-8966-4656-99d9-0c699734f1a8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.544200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.545351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.545860 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.546348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.547400 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:42.547930 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.550249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.559574 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 183/364] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:42.559574 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.592020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f53d31-de0c-402c-af56-28a46dbbbfd6 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.640753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969a916-3452-4ccf-8690-4361709d05ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.642306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a916-3452-4ccf-8690-4361709d05ec None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.642792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a916-3452-4ccf-8690-4361709d05ec None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.644134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a916-3452-4ccf-8690-4361709d05ec None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.646008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a916-3452-4ccf-8690-4361709d05ec None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.648617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a916-3452-4ccf-8690-4361709d05ec None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.652555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 182/365] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:42.655498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.655984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.656414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.681648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:42.682834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:42.684305 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:42.697170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d2986-cead-4d52-8981-19d1dd143f0e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.741367 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.742510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.742884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.743255 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.744726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.747593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.750371 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 184/366] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:42.751234 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.763110 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:42.778552 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 183/367] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:42.779447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.892308 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b116-0762-4c04-a9a9-b834ba072af3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.915079 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3cdb-dafb-4302-8620-2ece2e75aff4 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:42.939494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.941207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.943001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.943538 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.958481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:42.960841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:42.961779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:42.962228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:42.962679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:42.963703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:42.964170 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:42.965878 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:42.967707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:42.968997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:42.970156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:42.972290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 185/368] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:42.973752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:42.976198 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 184/369] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:42.977485 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.010571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.057359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.118026 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245a98c-509a-4751-b92a-88fa0968c471 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.156602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.157612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939345a0-6ffe-49ed-bedc-60e19ea232bc None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.158222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.158726 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.159249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.172539 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:43.181345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:43.182449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:34:43.183377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:43.186084 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 185/370] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:43.187179 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.219100 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.219942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.220235 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.220494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.223920 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:43.225390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:43.227093 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 186/371] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:42 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:43.227877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.314054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d54d76-bb1d-4c75-a522-5e2e6ac3d8a5 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.332283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.346621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.360366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.361180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.361584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.361934 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.362925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:43.363303 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:43.365074 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:43.368397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f356a-3d77-4cf8-8963-c34c62aee004 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.377629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 186/372] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:43.378602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.418434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.420074 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.420549 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.422047 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.423043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:43.423613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:43.425442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:43.438925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 187/373] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:43.440759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.525475 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08f58-fa29-4193-a55d-a97befd75a74 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.572600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edd3d4-d739-4350-ba7d-01981a69c292 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.574857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.576458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.576738 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.577022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.577910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=266c22fce804450d859a7e09765df901) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:43.578223 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:43.579713 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:43.588434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 187/374] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:43 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=266c22fce804450d859a7e09765df901&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:43.589389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.612432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.613163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.613474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.613748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.637824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.648447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.705275 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:43.708472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:43.736427 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e464e-5dcf-4f14-8a49-9809aae372ae None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:43.799646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45002-0823-44b3-8297-d20974cdffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:43.800472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45002-0823-44b3-8297-d20974cdffb0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.800747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45002-0823-44b3-8297-d20974cdffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.801019 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45002-0823-44b3-8297-d20974cdffb0 None admin] PATH_INFO: `/v3/users/8191d30b64f64a5389d1c955b9769553` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.806119 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45002-0823-44b3-8297-d20974cdffb0 None admin] RBAC: Authorizing `identity:get_user(user_id=8191d30b64f64a5389d1c955b9769553)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:43.807624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45002-0823-44b3-8297-d20974cdffb0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:43.809720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 188/375] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:34:43 2022] GET /identity/v3/users/8191d30b64f64a5389d1c955b9769553 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:43.811040 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19713996-0b06-4da8-8759-cf080d766892 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.811323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19713996-0b06-4da8-8759-cf080d766892 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.811593 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19713996-0b06-4da8-8759-cf080d766892 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.813076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 189/376] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:43.839192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 188/377] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:43.842941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:43.843258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:43.843538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:43.843666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:43.897941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:43.899524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:43.902185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:43.978583 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72005cc-543c-49fa-be83-c65b9f1980b0 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.022705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 190/378] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:44.024017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.061333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.062393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.063034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.063498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.064578 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] RBAC: Adding query filter params (role.id=82a463123e20467bb4c64ad6789ecfc8, user.id=8191d30b64f64a5389d1c955b9769553, scope.system=all) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:44.066832 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:44.068668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:44.094915 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:44.100119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 189/379] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:34:43 2022] GET /identity/v3/role_assignments?user.id=8191d30b64f64a5389d1c955b9769553&scope.system=all&role.id=82a463123e20467bb4c64ad6789ecfc8 => generated 510 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:44.101216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.209347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54113139-7cba-43f1-af06-7cd8e24d0e54 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.259273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.261781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.266267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.266879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] PATH_INFO: `/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.268550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] RBAC: Authorizing `identity:get_project(project_id=e7ef1b2ba8e64b46822150f5756f1d6e)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:44.270274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:44.275543 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 191/380] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e => generated 344 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:44.281490 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.362553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede7f65-8c50-40fc-9ac5-bb6be2f65190 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.373130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:44.406890 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e5d89-8993-401d-be9e-e0b57778c90a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.414853 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.415700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.415983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.416251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.417579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:44.435770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:44.449213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.450582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.451080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.451543 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] PATH_INFO: `/v3/groups/f3ba8722510a4ff283b5f262878368c6` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:44.452351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:44.453508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:44.454453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 190/381] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:44.461269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.463898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] RBAC: Authorizing `identity:get_group(group_id=f3ba8722510a4ff283b5f262878368c6)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:44.465652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:44.467903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 192/382] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/groups/f3ba8722510a4ff283b5f262878368c6 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:44.473915 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.590399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3d7ff-c6de-4943-857a-36cf0e515504 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.628611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.629336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd1fa6-9282-41b2-9264-8f4f2979bcb6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.629874 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.630360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.630932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.632096 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] RBAC: Adding query filter params (group.id=f3ba8722510a4ff283b5f262878368c6, role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=e7ef1b2ba8e64b46822150f5756f1d6e) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:44.632907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:44.634630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:44.641472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 193/383] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/role_assignments?group.id=f3ba8722510a4ff283b5f262878368c6&scope.project.id=e7ef1b2ba8e64b46822150f5756f1d6e&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:44.681939 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.682637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.682913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.683185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.683946 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] RBAC: Adding query filter params (name=member) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:44.684249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:44.685819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:44.695902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 191/384] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:44.701773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.752664 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:44.844828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef2ab2-d8b6-45f2-9518-ab89c16c2fa9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:44.886461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:44.887285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:44.887566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:44.887839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:44.889101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:44.890652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:44.892780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 194/385] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:44.898699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:44.952831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:45.045711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6863256-c0d3-4c7d-bf86-2864cb088471 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:45.090204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73478e15-86aa-440d-b233-80fd55e0b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:45.091109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478e15-86aa-440d-b233-80fd55e0b574 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:45.091393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478e15-86aa-440d-b233-80fd55e0b574 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:45.091671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478e15-86aa-440d-b233-80fd55e0b574 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:45.092878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73478e15-86aa-440d-b233-80fd55e0b574 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:45.094385 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73478e15-86aa-440d-b233-80fd55e0b574 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:45.096419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 192/386] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:44 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:45.102938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:45.256363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400469b2-7df4-4d69-a7f0-f4facb9e8125 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:45.258142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:45.298505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec6ab0-8649-47d8-bd68-41a065821b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:45.299936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec6ab0-8649-47d8-bd68-41a065821b73 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:45.300571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec6ab0-8649-47d8-bd68-41a065821b73 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:45.301399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec6ab0-8649-47d8-bd68-41a065821b73 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901/groups/d5898869301b456e98d1efea19644fb0/roles/6b466334835d44468e51d8a0bfed2e86` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:45.303642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec6ab0-8649-47d8-bd68-41a065821b73 None admin] RBAC: Authorizing `identity:create_grant(project_id=266c22fce804450d859a7e09765df901, group_id=d5898869301b456e98d1efea19644fb0, role_id=6b466334835d44468e51d8a0bfed2e86)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:45.305123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec6ab0-8649-47d8-bd68-41a065821b73 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:45.326662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 195/387] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:34:45 2022] PUT /identity/v3/projects/266c22fce804450d859a7e09765df901/groups/d5898869301b456e98d1efea19644fb0/roles/6b466334835d44468e51d8a0bfed2e86 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:45.890378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abed8f0-4f8e-4270-842b-25ad39ba15ba None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:45.890919 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abed8f0-4f8e-4270-842b-25ad39ba15ba None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:45.890919 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abed8f0-4f8e-4270-842b-25ad39ba15ba None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:45.892549 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 193/388] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:45.897986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:45.898422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:45.898519 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:45.917093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:45.918310 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:45.992986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 196/389] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:46.016845 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abed8f0-4f8e-4270-842b-25ad39ba15ba None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.142092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:46.151234 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abed8f0-4f8e-4270-842b-25ad39ba15ba None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:46.195429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:46.196255 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.196531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.196801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.207417 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:46.216158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:46.217326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:46.218672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 194/390] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:46.223750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.337774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:46.348210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2af7f-75e1-4922-a1ae-c0dee6dd6df8 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:46.390771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:46.391453 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.391727 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.391993 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.392793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:46.393029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:46.394465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d57395-7627-4fcb-9364-eb5a353c6f2c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:46.408861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 197/391] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:46.414637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.461069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb25463-e704-44c8-a7bc-e36f14db676a None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.461432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb25463-e704-44c8-a7bc-e36f14db676a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.461637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb25463-e704-44c8-a7bc-e36f14db676a None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.463180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 198/392] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:46.471996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.472572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.473076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.510402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:46.512311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:46.546003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbc11d-7eb9-4b8d-a9a2-e03cbbd22ffc None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:46.581428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:46.582635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.583077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.583515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.585024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:46.586726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:46.589001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 195/393] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:46 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:46.594850 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.633677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:46.705447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd423-7f56-4256-bc26-858d6e4f5a9a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:46.740536 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 199/394] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:46.745928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:46.747032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.747459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.747878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.753941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:46.755549 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:46.757887 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 196/395] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:46 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:46.763855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.782605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.885512 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:46.901773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:46.910588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5a5bb-08de-4227-9df6-b105a2c0fa4e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:46.910905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c20498-58f5-4933-b650-ea36818866e2 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:46.949357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386e8f8-fd8d-49d0-b578-66d05b07888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:46.950595 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e8f8-fd8d-49d0-b578-66d05b07888f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.951078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e8f8-fd8d-49d0-b578-66d05b07888f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.951534 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e8f8-fd8d-49d0-b578-66d05b07888f None admin] PATH_INFO: `/v3/roles/member` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.957575 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:46.958791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.959231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.959579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/groups/d5898869301b456e98d1efea19644fb0/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.962788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=31dbad7c8bdb4192999277e9f519bc7b, group_id=d5898869301b456e98d1efea19644fb0, role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:46.964734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:46.965139 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386e8f8-fd8d-49d0-b578-66d05b07888f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-6386e8f8-fd8d-49d0-b578-66d05b07888f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:46.974603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:46.975701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:46.976613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:46.979570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 197/396] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:46.985295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21ec6-fc7f-49e0-9649-32fcd57289a9 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.985514 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 200/397] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:34:46 2022] PUT /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/groups/d5898869301b456e98d1efea19644fb0/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:46.985879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21ec6-fc7f-49e0-9649-32fcd57289a9 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.986254 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21ec6-fc7f-49e0-9649-32fcd57289a9 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:46.987877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 198/398] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:46.989548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:46.994364 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:46.994750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:46.994927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.025484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:47.026691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:47.168574 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 199/399] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:47.181659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bf245-9e86-488f-a205-bd9a0eb7dac3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.214958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.247130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.247835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.248120 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.248395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.249157 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:47.249491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:47.250914 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:47.268594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 201/400] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:47.278974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.305010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:47.337285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:47.361023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5abdc6-1bac-446b-a638-3b6116a7db3e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.407224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.408614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.409186 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.409740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.427764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c972cb-8500-448d-bf3a-1fdd5874c42f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.433960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:47.451120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:47.452453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:47.453616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:47.456225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 200/401] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:47.462071 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.471551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.481270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.481270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.481270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.481270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:47.481270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:47.481270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 202/402] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:47 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:47.483412 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.565013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:47.593694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db335a2-8bfe-4305-8bd3-c8124906c2d9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.594091 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d5a69-becf-4d2e-a1e5-1bb80c90d877 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.630996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.632263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.632756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.633363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.634802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:47.636506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:47.638851 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 203/403] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:47 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:47.645003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.647039 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.647737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.648021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.648294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.649496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:47.649705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:47.651602 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:47.664763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 201/404] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:47.670164 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.676813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:47.761187 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46520e0-ea98-430e-9f87-719e8ee69dfe None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.780823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644e7ee-3e9c-4ca4-bb6f-933fc4b321e8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:47.800498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.802320 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.803127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.803670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/groups/d5898869301b456e98d1efea19644fb0/roles/6b466334835d44468e51d8a0bfed2e86` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.806980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] RBAC: Authorizing `identity:create_grant(project_id=31dbad7c8bdb4192999277e9f519bc7b, group_id=d5898869301b456e98d1efea19644fb0, role_id=6b466334835d44468e51d8a0bfed2e86)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:47.809645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:47.816511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09271a6b-b072-4668-be97-d4fbd3196878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:47.817391 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271a6b-b072-4668-be97-d4fbd3196878 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.817664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271a6b-b072-4668-be97-d4fbd3196878 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.817968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271a6b-b072-4668-be97-d4fbd3196878 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.819570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09271a6b-b072-4668-be97-d4fbd3196878 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:47.821166 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09271a6b-b072-4668-be97-d4fbd3196878 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:47.827520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 202/405] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:47 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:47.828932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d908c-d7d0-4e53-ab2a-be9a9be411f2 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.829226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d908c-d7d0-4e53-ab2a-be9a9be411f2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.829808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d908c-d7d0-4e53-ab2a-be9a9be411f2 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.832073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 204/407] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:34:47 2022] PUT /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b/groups/d5898869301b456e98d1efea19644fb0/roles/6b466334835d44468e51d8a0bfed2e86 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:34:47.837340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 203/407] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:47.837558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:47.837795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:47.837967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:47.839146 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:47.845497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:47.865899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:47.867264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:47.993721 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 204/408] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:48.016319 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.018992 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:48.058039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717f4d-fe81-490b-8250-e2649b37abeb None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.127584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.128493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.128760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.129030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.132778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:48.134397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:48.136165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 205/409] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:47 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:48.142089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.183509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209446d-54d2-41eb-b2b0-3458c6865731 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.222080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:48.232174 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabeda90-c4cb-44b5-8281-41e7aef4fb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.233717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabeda90-c4cb-44b5-8281-41e7aef4fb4f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.234556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabeda90-c4cb-44b5-8281-41e7aef4fb4f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.234981 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabeda90-c4cb-44b5-8281-41e7aef4fb4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.249363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabeda90-c4cb-44b5-8281-41e7aef4fb4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-aabeda90-c4cb-44b5-8281-41e7aef4fb4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:48.257326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:48.258511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:48.259609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:48.260717 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c37e4b-d4e4-4576-b853-f18101d1ec1a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.264021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 205/410] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:48.264645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b16db-6b68-462d-af2a-92a8a50785a9 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.264935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b16db-6b68-462d-af2a-92a8a50785a9 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.265213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b16db-6b68-462d-af2a-92a8a50785a9 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.270892 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 206/411] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:48.270892 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b16db-6b68-462d-af2a-92a8a50785a9 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.302507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde0826-8f92-4cff-8fef-056175a8d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.303465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde0826-8f92-4cff-8fef-056175a8d6f9 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.303743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde0826-8f92-4cff-8fef-056175a8d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.304014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde0826-8f92-4cff-8fef-056175a8d6f9 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901/groups/d5898869301b456e98d1efea19644fb0/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.306339 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde0826-8f92-4cff-8fef-056175a8d6f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=266c22fce804450d859a7e09765df901, group_id=d5898869301b456e98d1efea19644fb0, role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:48.307806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde0826-8f92-4cff-8fef-056175a8d6f9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:48.337268 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 206/412] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:34:48 2022] PUT /identity/v3/projects/266c22fce804450d859a7e09765df901/groups/d5898869301b456e98d1efea19644fb0/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:34:48.337268 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.337268 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.338724 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.380642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:48.382985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:48.384478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:48.411351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b16db-6b68-462d-af2a-92a8a50785a9 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.448979 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.449697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.449978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.450247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.451014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:48.451311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:48.452773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:48.464140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 207/413] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:48.469469 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.578742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 207/414] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:48.603439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.632585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc2a05-f7c1-41fd-ae91-da8d75bac91a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.677074 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:48.681614 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:48.682510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.683406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.683685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.683958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] PATH_INFO: `/v3/groups/f3ba8722510a4ff283b5f262878368c6` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.686243 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] RBAC: Authorizing `identity:get_group(group_id=f3ba8722510a4ff283b5f262878368c6)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:48.687861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:48.689869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 208/415] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:48 2022] GET /identity/v3/groups/f3ba8722510a4ff283b5f262878368c6 => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:48.695966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.799219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8cb47-6d19-4fb7-8cc5-35ea09725bdb None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.821804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8075e2-ed66-4257-8f47-6854971aa21f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:48.863081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.864029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.864312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.864593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] PATH_INFO: `/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.868265 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] RBAC: Authorizing `identity:get_project(project_id=e7ef1b2ba8e64b46822150f5756f1d6e)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:48.869793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:48.877371 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 209/416] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:48 2022] GET /identity/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:48.878687 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.893591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:48.894414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:48.894695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:48.894969 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:48.921871 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:48.930302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:48.931324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:48.932340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:48.943166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 208/417] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:48.948451 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:48.969722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:49.010090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732036b-c797-47f8-bc9e-899105072d46 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:49.079180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:49.080097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:49.080378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:49.080650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] PATH_INFO: `/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e/groups/f3ba8722510a4ff283b5f262878368c6/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:49.083743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] RBAC: Authorizing `identity:create_grant(project_id=e7ef1b2ba8e64b46822150f5756f1d6e, group_id=f3ba8722510a4ff283b5f262878368c6, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:49.085269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:49.112722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 210/418] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:34:48 2022] PUT /identity/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e/groups/f3ba8722510a4ff283b5f262878368c6/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:34:49.145209 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d91112-b717-47c2-bb84-15d9eef5a086 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:49.168051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:49.190711 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:49.191581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:49.191910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:49.192237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:49.193147 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:49.193546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:49.195458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:49.212671 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 209/419] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:49.219422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:49.356983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1f037-472d-44c4-a5a4-8d9caa0c42cf None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:49.381794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:49.400796 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:49.401472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:49.401753 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:49.402073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:49.472743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:49.474289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:49.625134 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 211/420] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:49.630474 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:49.761138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7902d-dab5-42fc-9c41-c0482871a98c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:49.808814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067ad16-80bb-4aab-a641-a4b9f0df3734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:49.809751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ad16-80bb-4aab-a641-a4b9f0df3734 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:49.810029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ad16-80bb-4aab-a641-a4b9f0df3734 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:49.810296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ad16-80bb-4aab-a641-a4b9f0df3734 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:49.811498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ad16-80bb-4aab-a641-a4b9f0df3734 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:49.812980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ad16-80bb-4aab-a641-a4b9f0df3734 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:49.814939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 210/421] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:49 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:49.820389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:49.852513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:49.871652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca188cd-696b-4ffc-9a62-410b3a1ccb0c None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:49.871998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca188cd-696b-4ffc-9a62-410b3a1ccb0c None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:49.872194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca188cd-696b-4ffc-9a62-410b3a1ccb0c None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:49.873860 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 211/422] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:49.878833 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:49.879114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:49.879388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:49.897624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:49.898805 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:49.946439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:49.969646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 212/423] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:49.993618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:49.996498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4640a4-566a-4a3d-9cbe-98413aa0e046 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:50.035354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:50.037379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.037903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.038360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.040210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:50.042046 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:50.044739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 212/424] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:49 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:50.050423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:50.165925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e0359-6608-4320-a55b-c99b7a4580d9 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:50.208492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:50.227938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b015b-3373-4b52-aaaa-44ca0f631ebb None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:50.231017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:50.231851 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.232124 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.232389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.251872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:50.261564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:50.262632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:50.263557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:50.264866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 213/425] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:50.270296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:50.300362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:50.301414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.301878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.302355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.303410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=266c22fce804450d859a7e09765df901) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:50.305427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:50.308830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:50.319794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 213/426] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:50 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=266c22fce804450d859a7e09765df901&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:50.363651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:50.442113 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbbbb2-8e85-4f76-a207-b4aec778f09f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:50.492830 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:50.493929 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.494396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.494892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.495960 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:50.496468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:50.498148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe44c37-3b39-4b7c-8f09-bd737fbc5ec6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:50.525322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 214/427] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:50.531137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:50.675622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be91b4-94b0-41cf-916b-4daeab68515e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:50.716926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:50.717650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.717933 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.718205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.747128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:50.809532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:50.811015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:50.848117 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3732b-742a-4b6c-82e6-e341d19c35f5 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.848451 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3732b-742a-4b6c-82e6-e341d19c35f5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.848824 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3732b-742a-4b6c-82e6-e341d19c35f5 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.850655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 215/428] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:50.857030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:50.857357 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:50.857606 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:50.884327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:50.885537 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:50.942253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 214/429] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:34:50.947941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:50.967834 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 216/430] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:50.996439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.016352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.031312 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.093411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6431077-6017-4eff-bc59-f24a1b0e9243 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.119060 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383150-ad58-4242-9a75-bda682a59f35 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.132147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59624934-d55a-44a7-93ac-af23607283f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.133477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59624934-d55a-44a7-93ac-af23607283f8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.133819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59624934-d55a-44a7-93ac-af23607283f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.134103 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59624934-d55a-44a7-93ac-af23607283f8 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.135942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59624934-d55a-44a7-93ac-af23607283f8 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:51.138188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59624934-d55a-44a7-93ac-af23607283f8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:51.141398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 215/431] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:50 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:51.141398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48943-52a5-4ed3-8b20-42a1c0388f7a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.141398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48943-52a5-4ed3-8b20-42a1c0388f7a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.141726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48943-52a5-4ed3-8b20-42a1c0388f7a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.143046 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 216/432] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:51.148356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd48943-52a5-4ed3-8b20-42a1c0388f7a None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.163307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a844-1a76-4a24-9c7f-8f319da6c923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.164121 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a844-1a76-4a24-9c7f-8f319da6c923 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.164393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a844-1a76-4a24-9c7f-8f319da6c923 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.164663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a844-1a76-4a24-9c7f-8f319da6c923 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.175544 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906a844-1a76-4a24-9c7f-8f319da6c923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c906a844-1a76-4a24-9c7f-8f319da6c923 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:51.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:51.185609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:51.186732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 217/433] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:51.190724 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.191854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.191854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.214173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:51.215459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:51.278766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd48943-52a5-4ed3-8b20-42a1c0388f7a None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.293309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 218/434] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:51.293309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.312100 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.315275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.329799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.330669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.330945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.331210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.332382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:51.333845 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:51.340530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 217/435] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:51.340530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.402050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e5361-e734-4429-8f42-687bc8d075c1 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.436742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.437456 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.437738 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.438015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.438762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] RBAC: Adding query filter params (name=member) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:51.439086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:51.440466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:51.442138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18992916-db66-4b39-b4fb-9d4844dd5723 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.448493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 219/436] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:51.450264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.479608 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.480370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.480654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.480913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.491925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:51.499960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:51.500983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:51.502017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 218/437] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:51.502814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.547791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.558781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.572542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e510a2-2c30-4ffb-bb3c-dcbb729bbca0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.615093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.616088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.616507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.616873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.618111 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:51.618572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:51.620354 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:51.626749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af8a0b-0351-414f-b919-3243b842d55b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.627578 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 220/438] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:51.628728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.686935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a1e3f-37e5-4db1-8621-1d085755468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.688082 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a1e3f-37e5-4db1-8621-1d085755468f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.688371 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a1e3f-37e5-4db1-8621-1d085755468f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.688651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a1e3f-37e5-4db1-8621-1d085755468f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.753902 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae07c0-57ca-40aa-8d36-44d179d9e3f4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.778466 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a1e3f-37e5-4db1-8621-1d085755468f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:51.780005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a1e3f-37e5-4db1-8621-1d085755468f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:51.796986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.797876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.798141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.798418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.799188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:51.799497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:51.800953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d8e3-a062-424b-9818-2bc595387c61 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:51.811068 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 221/439] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:51.812564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc556e-2569-4dd5-9ab6-62d2dc33dfc7 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.812857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc556e-2569-4dd5-9ab6-62d2dc33dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.813130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc556e-2569-4dd5-9ab6-62d2dc33dfc7 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.814654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 222/440] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:51.816689 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc556e-2569-4dd5-9ab6-62d2dc33dfc7 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:51.833894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.840958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:51.895467 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 219/441] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:51.896964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.897388 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.897665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.916683 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:51.917928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:51.934625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc556e-2569-4dd5-9ab6-62d2dc33dfc7 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:51.969223 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:51.969932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:51.970212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:51.970511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:51.989551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 220/442] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:51.990523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.039608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.041071 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.052109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.067950 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.089261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d1e05-f3d1-45a5-a4ec-24bdef421def None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.122462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.123327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.123598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.123863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] PATH_INFO: `/v3/projects/31dbad7c8bdb4192999277e9f519bc7b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.125074 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] RBAC: Authorizing `identity:get_project(project_id=31dbad7c8bdb4192999277e9f519bc7b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.126547 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.128234 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 221/443] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/projects/31dbad7c8bdb4192999277e9f519bc7b => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:52.132821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.143806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 223/444] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:34:52.143806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.228895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047a65d-481b-4b8f-a29d-063750f27b60 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.243920 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a118-87a9-4474-bdaf-6ef29dffcc39 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.255132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.264270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.265080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.265395 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.265664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.276863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:52.279980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.280848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.281128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.281436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.282612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.284081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:52.284483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:52.285529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:34:52.286468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:52.287239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 224/445] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:52.287239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.288099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 222/446] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:52.291547 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.300252 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.387340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22578b9b-528e-4639-beb2-8007dcec0344 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.390382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bf7b4-e47c-4881-bd34-c5ca4f1db85f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.420285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.421161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.421473 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.421741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] PATH_INFO: `/v3/projects/266c22fce804450d859a7e09765df901` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.422939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] RBAC: Authorizing `identity:get_project(project_id=266c22fce804450d859a7e09765df901)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.423341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.424007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.424285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.424389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.424558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.425330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:52.425635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.426128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 225/447] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/projects/266c22fce804450d859a7e09765df901 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:52.426891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.427001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.439965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 223/448] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:52.439965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.477373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.507715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.536652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0940b-4511-492b-81ab-86945bd6bbc3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.543676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a4e3-588b-49b7-a864-8aecdd568529 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.569563 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.570409 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.570912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.571231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.572070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=6b466334835d44468e51d8a0bfed2e86, scope.project.id=31dbad7c8bdb4192999277e9f519bc7b) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:52.572598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.575065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.581481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.581995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 226/449] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=31dbad7c8bdb4192999277e9f519bc7b&role.id=6b466334835d44468e51d8a0bfed2e86 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:52.582357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.582631 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.582731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.582897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] PATH_INFO: `/v3/groups/d5898869301b456e98d1efea19644fb0` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.584154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] RBAC: Authorizing `identity:get_group(group_id=d5898869301b456e98d1efea19644fb0)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.585694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.587374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 224/450] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/groups/d5898869301b456e98d1efea19644fb0 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:52.593516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:52.703149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47af3d-55d2-4193-bad2-1d2235e52fb3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.711014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8eff1-9c3c-44b1-86aa-5de27b0489a8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:52.715130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.748073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.748729 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.749010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.749304 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.752031 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:52.752936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:52.753388 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:52.753777 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:52.754653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] RBAC: Adding query filter params (group.id=d5898869301b456e98d1efea19644fb0, role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=266c22fce804450d859a7e09765df901) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:52.754961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.756504 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.759958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:52.773542 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 225/451] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/role_assignments?group.id=d5898869301b456e98d1efea19644fb0&scope.project.id=266c22fce804450d859a7e09765df901&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:34:52.845649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:52.847104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:52.952569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 227/452] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:34:52.958447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:53.061962 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcf0cb-2559-4771-8b81-abd6c587cc95 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:53.095530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a2771-6dca-4523-a245-355da18167ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:53.096500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a2771-6dca-4523-a245-355da18167ac None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:53.096690 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a2771-6dca-4523-a245-355da18167ac None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:53.096988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a2771-6dca-4523-a245-355da18167ac None admin] PATH_INFO: `/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:53.098259 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a2771-6dca-4523-a245-355da18167ac None admin] RBAC: Authorizing `identity:get_project(project_id=e7ef1b2ba8e64b46822150f5756f1d6e)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:53.099803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a2771-6dca-4523-a245-355da18167ac None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:53.101523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 226/453] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:34:52 2022] GET /identity/v3/projects/e7ef1b2ba8e64b46822150f5756f1d6e => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:53.107454 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:53.121213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:53.210683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216771c-33e9-4212-bef5-03a5c6720781 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:53.245028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c6e81-6831-4f4d-9fc4-cf85bab49ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:53.245938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c6e81-6831-4f4d-9fc4-cf85bab49ba4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:53.246224 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c6e81-6831-4f4d-9fc4-cf85bab49ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:53.246496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c6e81-6831-4f4d-9fc4-cf85bab49ba4 None admin] PATH_INFO: `/v3/groups/f3ba8722510a4ff283b5f262878368c6` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:53.247687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c6e81-6831-4f4d-9fc4-cf85bab49ba4 None admin] RBAC: Authorizing `identity:get_group(group_id=f3ba8722510a4ff283b5f262878368c6)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:53.249150 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c6e81-6831-4f4d-9fc4-cf85bab49ba4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:53.251214 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 228/454] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:34:53 2022] GET /identity/v3/groups/f3ba8722510a4ff283b5f262878368c6 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:53.256991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a2771-6dca-4523-a245-355da18167ac None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:53.331101 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:53.366038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a2771-6dca-4523-a245-355da18167ac None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:53.400274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:53.400939 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:53.401221 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:53.401568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:53.402402 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] RBAC: Adding query filter params (group.id=f3ba8722510a4ff283b5f262878368c6, role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=e7ef1b2ba8e64b46822150f5756f1d6e) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:53.402714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:53.404169 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25a62-0356-494b-85ef-1dc81668455e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:53.410244 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 227/455] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:34:53 2022] GET /identity/v3/role_assignments?group.id=f3ba8722510a4ff283b5f262878368c6&scope.project.id=e7ef1b2ba8e64b46822150f5756f1d6e&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:56.168244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92663-4ceb-43e5-a8fe-797a51a6b462 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.168822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92663-4ceb-43e5-a8fe-797a51a6b462 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.168822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92663-4ceb-43e5-a8fe-797a51a6b462 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.170460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 229/456] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:56.176697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.177285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.177285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.202226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:56.203451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:56.251098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a19be5-a8db-4265-a9ec-d7ae8702d7ed None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.251789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a19be5-a8db-4265-a9ec-d7ae8702d7ed None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.252272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a19be5-a8db-4265-a9ec-d7ae8702d7ed None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.255348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 230/457] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:56.260230 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.260825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.261344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.291783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:56.293031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:56.305178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 228/458] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:56.369269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:56.424198 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 231/459] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:34:56.452613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:56.465773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:56.478640 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:56.531493 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215208d-48dc-42af-83d6-91dacbae2ef2 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:56.579928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:56.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.581032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.581337 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.595278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:56.597479 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:56.602230 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223c3b1-1e1c-4b98-b265-8486cb78ceec None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-5b7fd93e-7cbd-4a78-aa01-2dbed86962ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:34:56.624732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:56.625971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:56.627357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:34:56.627357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:34:56.627357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:56.627357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:56.627357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 229/460] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:56.627703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0be5-b447-4093-8fc1-7d4ff76afe4d None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.628000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0be5-b447-4093-8fc1-7d4ff76afe4d None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.628171 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e0be5-b447-4093-8fc1-7d4ff76afe4d None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.637690 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 230/461] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:56.638212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0be5-b447-4093-8fc1-7d4ff76afe4d None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:56.658853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:56.660075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.660595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.661097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.672762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:56.674646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9e4df352-d254-4e2f-afb6-89fc10489514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:34:56.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:56.694315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:56.695516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:34:56.695516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:34:56.695516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:56.695516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:56.697618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 232/462] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:34:56.699158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.699643 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.700112 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.724911 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:56.726755 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:56.775546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e0be5-b447-4093-8fc1-7d4ff76afe4d None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:56.794632 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:56.796667 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:56.814779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 233/463] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:56.816101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:56.817680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:56.818362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.818653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.818926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.820175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:56.820485 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:56.821965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:56.834130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 231/464] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:56.840030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:56.944709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b97e8-51da-4379-8a8f-b357c14b8f5e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:56.952677 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf06b17-b0e5-486b-81aa-44cbda79f0fc None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:56.978611 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:56.979296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:56.979583 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:56.979858 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:56.980575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:56.982118 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:56.999478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.001653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.003356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.003629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.004377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:57.004680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.006139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.010553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 232/465] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:56 2022] POST /identity/v3/users => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:57.011856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.034100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 234/466] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:57.039699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.052537 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.108980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.135980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db2fd-c618-4838-9abb-7ff7e4dbfaee None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.168615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fe644-feb8-4fef-afa0-de7095213213 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.177490 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.178969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.179456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.179893 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.195228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.197229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.216625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c5888-a23d-4eec-a3dd-57971ae798d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.217607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c5888-a23d-4eec-a3dd-57971ae798d7 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.217988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c5888-a23d-4eec-a3dd-57971ae798d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.218367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c5888-a23d-4eec-a3dd-57971ae798d7 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.219262 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c5888-a23d-4eec-a3dd-57971ae798d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-1efeb07a-fb2e-4fcb-869e-af060792ff7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:34:57.220927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:57.222130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:57.223222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:34:57.223222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:34:57.223222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:57.223222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:34:57.223470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c5888-a23d-4eec-a3dd-57971ae798d7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.225854 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 233/467] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:57.227687 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d87e1-b2a1-4b26-9eb9-db4f666ad317 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.228146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d87e1-b2a1-4b26-9eb9-db4f666ad317 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.228570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d87e1-b2a1-4b26-9eb9-db4f666ad317 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.230535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 234/468] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:57.233291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d87e1-b2a1-4b26-9eb9-db4f666ad317 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.258253 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 235/469] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:57 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:57.259667 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea4c11-e7cc-4d4b-b228-661c8c91616f None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.259987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea4c11-e7cc-4d4b-b228-661c8c91616f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.260363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea4c11-e7cc-4d4b-b228-661c8c91616f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.279395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-15ea4c11-e7cc-4d4b-b228-661c8c91616f None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:57.280792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-15ea4c11-e7cc-4d4b-b228-661c8c91616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:57.334048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.369560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d87e1-b2a1-4b26-9eb9-db4f666ad317 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.379347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 236/470] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:57.411315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac4e85-d1bc-48f4-a477-92f3efeb2fc5 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.411656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac4e85-d1bc-48f4-a477-92f3efeb2fc5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.411883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac4e85-d1bc-48f4-a477-92f3efeb2fc5 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.413812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 237/471] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:34:57.414392 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac4e85-d1bc-48f4-a477-92f3efeb2fc5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.417631 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.418989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.418989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.418989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.419664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:57.419949 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.421510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb41acb-7ae1-4ee9-aa23-a5a0d6c1cd99 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.435513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 235/472] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:57.437666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.438191 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.438680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.459455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:34:57.460669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:34:57.465837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.550010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 236/473] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:34:57.551404 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.563808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac4e85-d1bc-48f4-a477-92f3efeb2fc5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.613598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.614435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.614774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.615026 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.627062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.628657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.646239 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:34:57.649007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:57.650628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:57.651690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:34:57.651690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:34:57.651690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:34:57.651690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:57.651690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 238/474] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:57.651996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.691602 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669f02-0ff8-4a53-a293-76065e309c94 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.731441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.732046 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.732325 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.732596 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.733347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.734822 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.761177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 237/475] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:57 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:34:57.761995 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.779432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.796403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521e5ca-9a7e-494f-bc04-f215a494d2e0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.836448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.837192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.837516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.837795 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.838662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.840194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.853814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.sql.core [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:34:57.853814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:34:57.853814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [parameters: {'id': '0cc07ca038a3476d84f985c6b3e1a129', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:34:57.853814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application conn.query(q) May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result.read() May 19 14:34:57.882811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:34:57.883901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application next(self.gen) May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application yield resource May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application next(self.gen) May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise self.value May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application session.commit() May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:34:57.884978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.session.flush() May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application compat.raise_( May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:34:57.886030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:34:57.887056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application conn.query(q) May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result.read() May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:34:57.888062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [parameters: {'id': '0cc07ca038a3476d84f985c6b3e1a129', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:57.888977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:34:57.896422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:57.899660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 239/476] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:34:57 2022] POST /identity/v3/roles => generated 144 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:57.899869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f66c2-239c-49d1-ae53-9cd0e5c7b9f6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:57.904954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:57.926138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:57.938517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:57.940032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:57.940546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:57.941209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:57.944968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:57.945780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:57.949396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:57.976852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 238/477] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:57.983491 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:58.108798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cfc42-e108-410b-bccc-1ec0ee74ca06 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:58.126238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c7930-0101-48ef-8748-80c924c3e179 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:58.152762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:58.154437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:58.155083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:58.155665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:58.156848 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:58.159242 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:58.169929 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b290f0-6136-4097-b79e-d9f2fcdb8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:58.172238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b290f0-6136-4097-b79e-d9f2fcdb8e7b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:58.172358 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b290f0-6136-4097-b79e-d9f2fcdb8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:58.172616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b290f0-6136-4097-b79e-d9f2fcdb8e7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:58.188020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b290f0-6136-4097-b79e-d9f2fcdb8e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:34:58.188837 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 239/478] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:34:57 2022] POST /identity/v3/users => generated 249 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-a6b290f0-6136-4097-b79e-d9f2fcdb8e7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:34:58.197921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:34:58.199081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:34:58.200009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 240/479] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:34:58.205417 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:34:58.259522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:34:58.332059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ab93-336d-457e-b119-2b48af048b10 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:34:58.381199 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:34:58.382109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:58.382487 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:58.382855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:58.383975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:34:58.384420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:34:58.386570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60fdc5-d973-4159-b071-6934683092a0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:34:58.404054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 240/480] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:34:59.981894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618fa5e-cbb5-4595-8ac1-f39fda2f0c48 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:59.982533 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618fa5e-cbb5-4595-8ac1-f39fda2f0c48 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:59.982533 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618fa5e-cbb5-4595-8ac1-f39fda2f0c48 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:34:59.984130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 241/481] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:34:59.989762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540ec0b-3703-4c3a-ab06-bf58b90c4de6 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:34:59.990196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540ec0b-3703-4c3a-ab06-bf58b90c4de6 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:34:59.990307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540ec0b-3703-4c3a-ab06-bf58b90c4de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.014623 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-5540ec0b-3703-4c3a-ab06-bf58b90c4de6 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:00.015835 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-5540ec0b-3703-4c3a-ab06-bf58b90c4de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:00.115125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 241/482] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:00.148765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618fa5e-cbb5-4595-8ac1-f39fda2f0c48 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:00.175760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:00.293158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618fa5e-cbb5-4595-8ac1-f39fda2f0c48 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:00.300478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be947304-ce5a-48d9-8ede-5df3c4b1814b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.300759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be947304-ce5a-48d9-8ede-5df3c4b1814b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.301041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be947304-ce5a-48d9-8ede-5df3c4b1814b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.302723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 242/483] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:00.307879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.308167 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.308445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.345528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:00.347867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:00.365142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:00.366374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.366842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.367297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.381854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:00.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:00.391423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:00.392389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:00.395023 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 242/484] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:00.401033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:00.416273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:00.499386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 243/485] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:00.539449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:00.547481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:00.617580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a691fb8-4ee9-4044-b894-b3e18c7e3454 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:00.691016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:00.691711 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.691991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.692264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.693127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:00.693597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:00.695095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac29097-ec01-46f6-9818-81cf46af7af9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:00.696674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7e842-1ee9-449a-81ae-342b0ab53845 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:00.716396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 243/486] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:00.717859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757f39-3627-47df-88cf-7387b537f33b None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.718140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757f39-3627-47df-88cf-7387b537f33b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.718415 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757f39-3627-47df-88cf-7387b537f33b None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.723577 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 244/487] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:00.724362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce757f39-3627-47df-88cf-7387b537f33b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:00.735239 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed6b42-e9d2-4130-ad0e-26aa277def50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:00.736069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed6b42-e9d2-4130-ad0e-26aa277def50 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.736814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed6b42-e9d2-4130-ad0e-26aa277def50 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.737083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed6b42-e9d2-4130-ad0e-26aa277def50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.751276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed6b42-e9d2-4130-ad0e-26aa277def50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-7bed6b42-e9d2-4130-ad0e-26aa277def50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:00.762062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:00.763117 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:00.764048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:00.764830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 244/488] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:00.766069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.769261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.769261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.796512 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:00.797952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:00.822718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:00.860064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce757f39-3627-47df-88cf-7387b537f33b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:00.874045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 245/489] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:00.882249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:00.900441 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:00.901290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:00.901576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:00.901850 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:00.904113 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:00.916215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:00.917750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:00.940480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:00.942428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:00.944234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:00.944234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:00.944234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:00.944234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:00.944234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 245/490] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:00.944234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.001099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197afa92-9b78-4ed6-982c-261f8ded3458 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.044005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.044698 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.044975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.045274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.046040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:01.046343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.047763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.061644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 246/491] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:01.061644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.061944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802150-3036-407c-8f09-ed51d9b0b706 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.082217 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.099046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07754aa9-337e-4b6b-b76f-2cde241d0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.099865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07754aa9-337e-4b6b-b76f-2cde241d0d1d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.101100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07754aa9-337e-4b6b-b76f-2cde241d0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.101100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07754aa9-337e-4b6b-b76f-2cde241d0d1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.111003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07754aa9-337e-4b6b-b76f-2cde241d0d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-07754aa9-337e-4b6b-b76f-2cde241d0d1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:01.118777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:01.119826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:01.120781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 246/492] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:01.122471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99968de6-ec1e-4de1-9f6a-0cd96ccdefb0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.122471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99968de6-ec1e-4de1-9f6a-0cd96ccdefb0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.122471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99968de6-ec1e-4de1-9f6a-0cd96ccdefb0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.125359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 247/493] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:01.125359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99968de6-ec1e-4de1-9f6a-0cd96ccdefb0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.176983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f1f5-d6af-41e2-8b5e-9ba06d55f3e4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.182016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.215204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.215897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.216180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.216453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.217180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:01.217516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.218921 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.230959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 247/494] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:01.231769 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.249463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99968de6-ec1e-4de1-9f6a-0cd96ccdefb0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.283891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.284697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.284975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.285259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.296199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.297702 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.309351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8bc4ded7-7b3b-4e60-88f1-798edfeef334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:01.314912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:01.315961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:01.316955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:01.316955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:01.316955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:01.316955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:01.316955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 248/495] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:01.317407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.317657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.317933 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.333201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097696a-7407-45f3-98b8-d6fac3b4e4ef None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.334712 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:01.335868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:01.368211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.368895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.369169 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.369473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.370263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:01.370566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.371959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02d491-5c65-4b90-823a-6257c5cb9f0c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.379996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 248/496] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:01.381204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4249eb-37ab-4e11-b256-95f34750f8a2 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.381515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4249eb-37ab-4e11-b256-95f34750f8a2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.381786 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4249eb-37ab-4e11-b256-95f34750f8a2 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.382996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 249/497] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:01.385338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4249eb-37ab-4e11-b256-95f34750f8a2 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.409729 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 249/498] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:01.410708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.438672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.499756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4249eb-37ab-4e11-b256-95f34750f8a2 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.511091 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0419-afbc-4b57-8e81-e9c9841632af None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.533409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.534211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.534488 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.534763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.542633 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.544076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.546363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.546868 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.547006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.547303 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.547560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.548294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:01.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.550375 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-1383a2f1-d266-4ba0-8fd3-4755eb52ee1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:01.560982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:01.562397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:01.563801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:01.564417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 250/499] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:01.564496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 250/500] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:01.564496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.564496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.564496 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.566370 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.583021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:01.584233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:01.651940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 251/501] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:01.652776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.669832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.683205 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54c636-990d-4fed-a16d-83f89a7ccbdf None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.717005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.717841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.718123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.718402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.729263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.730825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:01.748448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:01.749631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:01.750858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:01.750858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:01.750858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:01.750858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:01.753377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 251/502] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:01.754340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.754781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a12df-97dd-4aa3-ab56-844c5907b2a0 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.783081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.788899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.789746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.790022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.790289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.800291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:01.808091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:01.809374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:01.810558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:01.814273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 252/503] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:01.814936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.859410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3d19-e324-44f3-9b33-0963ad0ab579 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.893783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.894583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.894858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.895125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.903131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.904607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.907157 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:01.916093 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b0da-226c-4420-90cf-7f0c0d8b46b1 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:01.921566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:01.922746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 252/504] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:01.923935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:01.952989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:01.953704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:01.953975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:01.954251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:01.955098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:01.955407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:01.956875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:01.993940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 253/505] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:01.994788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.029395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dc53c-88b1-4cfa-aa01-03412baa3b89 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.048573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.063354 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.064161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.064440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.064713 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.076364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.077903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:02.095436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:02.096641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:02.097710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:02.097710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:02.097710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:02.097710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:02.097710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 253/506] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:02.097710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.110966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ee3a8-307a-4dcd-89c9-57f3eed107df None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.147776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.148445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.148721 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.148980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.149662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.150096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:02.150400 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.151903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.163530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 254/507] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:02.164321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.200715 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544678f-8b82-4840-8afc-2f0ee5d8e0e1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.234295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.234971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.235250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.235525 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.236277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:02.236591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.238367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.247100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 254/508] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:02.247883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.267862 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398d25-c560-4678-8de5-5e145cac01d8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.277869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.302678 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.303450 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.303740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.304012 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.304847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:02.305151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.306612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.324124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 255/509] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:02.324928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.347470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae971b3f-2c52-4324-a3a1-bfb825f87e21 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.362511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.383061 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.383716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.383993 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.384265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.423738 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405b9c5-1349-495c-a91f-96437a06051d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.450474 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.452009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.457667 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.458333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.458609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.458882 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.459622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:02.459928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.461317 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.472577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 256/510] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:02.473429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.499968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.556045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 255/511] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:02.556895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.573372 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.579794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56615e4-0266-4caf-9122-23c52815b583 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.613655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.614412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.614700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.615037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.623076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.624807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:02.639482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:02.640529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 257/512] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:02.641688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.663426 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4aa66-ddff-47d6-8fec-1caf70c32289 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.697376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.698198 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.698473 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.698741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.708466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.710020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:02.727064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:02.728042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:02.729065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:02.729065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:02.729065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:02.729065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:02.729065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 256/513] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:02.729065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.730231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.746155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b6819-d9f5-4468-bff6-d1e617af4099 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.781660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.782368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.782645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.782927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.793462 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.828527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5f9dc-2459-4eab-8856-446231fa0a83 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:02.850577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.852075 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.863131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:02.863857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:02.864077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:02.864340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:02.865029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:02.866556 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:02.886560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 257/514] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:35:02 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:02.887380 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:02.946646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:02.966715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 258/515] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:02.972097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.019835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e552b-42ab-4932-9fca-f3f233b8d767 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.047137 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.064362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.065447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.065796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.066186 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.078354 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.080252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.091062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6380ca-9cbc-4c50-9a8b-97b612347f1d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:03.103038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:03.104056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:03.105523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:03.105523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:03.105523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:03.105523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:03.105523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 258/516] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:03.105523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.134749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.135472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.135762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.136035 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.136882 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:03.137191 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.138682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.157479 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 259/517] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:35:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:03.158403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.206576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.220765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c531f4e-3dc8-4553-a101-b134d0114e45 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.255232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.255910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.256188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.256511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.257285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:03.257621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.259029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.266696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a0f3-2239-428d-8c9e-b09f6703980c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.271040 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 259/518] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:03.271872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.305315 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.306016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.306296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.306593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.312055 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.317649 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.319219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:03.338390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:03.339429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:03.341328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:03.341328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:03.341328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:03.341328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:03.341328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 260/519] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:03.341328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.387010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfae265-ac1f-4d21-bf84-7f0e9caa2d48 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.421754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.422441 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.422722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.422996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.423730 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:03.424032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.425822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.439128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 260/520] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:03.440558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.444747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.451840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b4629-61c7-47e0-853a-c4efd87d2dec None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.485225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.485912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.486190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.486461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.546038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.553373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bcdc4-bb60-4b23-8185-b98cfc7c452c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.571049 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.572537 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.590629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.591432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.591728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.592157 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.600487 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.602014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:03.616695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:03.617807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:35:03.618853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:03.621687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 261/521] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:03.622785 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.675409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.682870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 261/522] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:03.683891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.731919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac0b2c-1433-4919-930b-6db6f3519983 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.767637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.768573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.769018 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.769317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.770044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:03.770346 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.771792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.788974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 262/523] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:03.788974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.794719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1cda5-8c6b-4699-8efa-3e973a161fba None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.802884 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.830593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.831392 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.831671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.831941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.840348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:03.855470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:03.856592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 262/524] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:03.857794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:03.898794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c7a2-5227-442d-9f06-df086c46fa0e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.926064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:03.936853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:03.938047 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:03.938461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:03.938882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:03.939727 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:03.940183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:03.942377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:03.971691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8619d-10d5-4cde-af06-239c7eb51307 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:03.982407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 263/525] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:03.983234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.012184 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.013011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.013310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.013575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.025090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.026584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:04.044998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:04.046301 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:04.047559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:04.047559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:04.047559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:04.047559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:04.049316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 263/526] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:04.054176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.058811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.091592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60897f-6a0d-46f4-bbc1-288d41afb3f1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.127584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.128400 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.128682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.128956 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.136092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.137607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:04.152680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:04.153739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:04.154743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:04.158683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 264/527] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:04.159536 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.169849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2827ed-b714-401f-8127-13342e4760b9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.180551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.208424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.209141 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.209459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.209766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.210576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:04.210879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.212365 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.232285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 264/528] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:04.233115 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.275164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d3e8-ab2a-40e7-85fb-87a525465b1d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.306992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.312565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.313264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.313583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.313861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.342740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1318811-54a9-454c-bcba-6b8d6252b1e7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.381403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.382083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.382369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.382652 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.383381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:04.383690 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.385029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.385163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.386841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.396909 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 265/529] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:04.397805 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.432087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.497350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 265/530] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:04.498311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.519025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b697d-86a8-4446-8606-b807a809a2b7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.538850 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.556156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.556833 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.557114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.557418 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.558216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:04.558528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.559999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.572743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 266/531] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:04.573491 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.618559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7fc1f-5e6d-402a-8702-59bc2f9272c2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.656439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.657095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.657411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.657675 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.658512 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=f5c6cd73a1234dde953e73fedc6440ee) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:04.658821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.660382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.672672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 266/532] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/role_assignments?user.id=f5c6cd73a1234dde953e73fedc6440ee&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:04.672672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.675395 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.687542 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1805c-0828-4500-b3e4-4107720148c7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.729421 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.730247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.730525 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.730798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.740711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.744088 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:04.768876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:04.770009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 267/533] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:04.777795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.805981 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:04.813402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4f9b3-92bb-450b-a6b5-e651a0b37633 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.857012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.858918 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.859445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.860116 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.875767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.878014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:04.902190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:04.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:04.904220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:04.904220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:04.904220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:04.904220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:04.904463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73b88-b9aa-403f-b398-6b46ebd9f68e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:04.907320 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 267/534] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:04.908403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.948318 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:04.949555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:04.950897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:04.951357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:04.952359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=02409707922c4ddf9f8c896c13ef651a) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:04.952827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:04.954554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:04.963297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 268/535] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/role_assignments?user.id=02409707922c4ddf9f8c896c13ef651a&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:04.964770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:04.988880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:05.038517 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3082095-e8bf-4337-b8ca-eb760f9165e3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.079437 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.080377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.080661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.081035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.082018 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:05.082429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:05.084302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:05.109266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 268/536] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:05.110564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc1c7e-1a56-4f6c-b8ce-31d4c2df3c98 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.113156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:05.127369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:05.180882 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.181632 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.181905 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.182183 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.183094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:05.183471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:05.185085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8156e-73b8-4c9e-8930-e85bc3f16114 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:05.199643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 269/537] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:05.200897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea9697-70f9-4c3d-a094-8ffc3acadacb None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.201187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea9697-70f9-4c3d-a094-8ffc3acadacb None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.206230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea9697-70f9-4c3d-a094-8ffc3acadacb None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.207991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 270/538] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:05.208812 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea9697-70f9-4c3d-a094-8ffc3acadacb None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:05.249837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8eff-c732-4a38-8c93-6afe99baf7d2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.290701 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.291543 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.291951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.292225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.293373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=c704d1a1b11e44a6889504246016996a) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:05.293690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:05.294979 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:05.297125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89cb67-6588-46f5-b66e-3d451e2cc3b1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:05.305709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 269/539] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/role_assignments?user.id=c704d1a1b11e44a6889504246016996a&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:05.307033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.307325 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.307603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.346534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:05.347866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:05.360082 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea9697-70f9-4c3d-a094-8ffc3acadacb None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.439242 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.439522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.439791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.452248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:05.454674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:05.479952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:05.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:05.481999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 271/540] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:05.492049 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 270/541] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:05.492807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:05.520571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:05.523601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:05.661579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713f436-0f80-4168-97b4-c34a6ca01202 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.701136 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.702005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.702342 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.702555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.714767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:05.717028 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a52c7e-33ed-4097-a1c2-1e98a6f0ebbb None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:05.724151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:05.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:05.726248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 271/542] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:05.734718 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:05.774502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:05.782453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.783136 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.783414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.783688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.784484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:05.784790 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:05.787028 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:05.809747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 272/543] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:05.815234 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:05.837413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:05.899721 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ffa60-bd39-4a1b-b2ec-45cb8625aa20 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:05.959062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:05.959732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:05.960016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:05.960292 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:05.961056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:05.961391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:05.962848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:05.991633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 272/544] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:05.993120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208da4c-b313-4698-a11e-ba990c1cddc9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:06.002269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:06.037080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:06.038490 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:06.039021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:06.039550 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:06.040636 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=bafd4f61582941a88f68e01deb85d4a7) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:06.041161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:06.042859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:06.049583 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 273/545] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/role_assignments?user.id=bafd4f61582941a88f68e01deb85d4a7&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:06.206067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd645f43-8fbe-4b65-8016-5d4747323f15 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:06.240145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:06.260877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:06.261744 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:06.262029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:06.262306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:06.275481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:06.277022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:06.306228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:06.307333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:06.308338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:06.308338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:06.308338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:06.308338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:06.308338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 273/546] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:06.313504 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:06.404164 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:06.464813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855dcef7-5f26-429e-962b-31c9d4bee6f3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:06.501603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:06.502303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:06.502584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:06.502857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:06.503581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:06.503888 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:06.505298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:06.519285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 274/547] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:06.525263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:06.646446 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe5a1c-252d-4640-b23e-b20b5e36d813 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:06.692681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:06.693529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:06.693835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:06.694112 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:06.706358 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:06.707866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:06.724501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:06.725746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:06.726807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 274/548] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:06.732306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:06.858036 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748975cc-39ea-42d8-abcc-db9b59b71f62 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:06.878529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:06.897820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:06.898505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:06.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:06.899401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:06.900248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:06.900556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:06.902034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:06.928974 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 275/549] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:35:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:06.934528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:06.982442 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:07.079442 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c450b6-08a3-47a4-8816-1cfa14407b04 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:07.117587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb67ea-3cd7-4b19-bf42-311e3fb60535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:07.118192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb67ea-3cd7-4b19-bf42-311e3fb60535 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:07.118468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb67ea-3cd7-4b19-bf42-311e3fb60535 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:07.118737 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb67ea-3cd7-4b19-bf42-311e3fb60535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:07.193800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb67ea-3cd7-4b19-bf42-311e3fb60535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:07.195368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb67ea-3cd7-4b19-bf42-311e3fb60535 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:07.265609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:07.329519 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 275/550] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:07.335456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:07.424916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484c30-463f-4bc7-b8a9-686f98583a17 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:07.425290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484c30-463f-4bc7-b8a9-686f98583a17 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:07.425533 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484c30-463f-4bc7-b8a9-686f98583a17 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:07.427016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 276/551] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:07.435325 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:07.435655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:07.435888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:07.469037 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:07.470381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:07.493798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce2d6e-8193-4857-8ae6-04ce8130f615 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:07.540548 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:07.541790 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:07.542283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:07.542773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:07.567747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:07.572280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:07.611975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:07.613078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:07.614349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:07.614349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:07.614349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:07.614349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:07.621377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 277/552] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:07.623908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 276/553] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:07.629257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:07.694849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:07.749687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:07.755888 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:07.761819 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6f6d8-9086-47d9-a0a9-057948401e84 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:07.798774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:07.799835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:07.800328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:07.800810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:07.801829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:07.802366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:07.804012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:07.818435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 278/554] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:07.826229 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:07.886917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549360-299a-4b64-9625-8c86e807dece None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:07.950362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bc6d9-6471-4c5b-a139-3e911163c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:07.951104 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bc6d9-6471-4c5b-a139-3e911163c8bf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:07.951375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bc6d9-6471-4c5b-a139-3e911163c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:07.951637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bc6d9-6471-4c5b-a139-3e911163c8bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:07.974769 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abfba3-fd3f-442e-acb0-5954d27ce479 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:07.993898 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bc6d9-6471-4c5b-a139-3e911163c8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:08.002002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-261bc6d9-6471-4c5b-a139-3e911163c8bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:08.010146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:08.011203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:08.012151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:08.015986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 277/555] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:08.018153 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da3a5c-3e0b-4d0e-9dd1-73fe4e35e1f7 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.018643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da3a5c-3e0b-4d0e-9dd1-73fe4e35e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.019102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da3a5c-3e0b-4d0e-9dd1-73fe4e35e1f7 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.021313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 278/556] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:08.023555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da3a5c-3e0b-4d0e-9dd1-73fe4e35e1f7 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.030004 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.030854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.031135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.031405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.054640 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.057464 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9926d8ef-5c22-4e40-b5f5-5a163a63e803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:08.082469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:08.083526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:08.084836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:08.088609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 279/557] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:08.090100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.090478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.090903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.125517 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:08.126812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:08.194411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.225498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 280/558] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:08.227251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.231415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da3a5c-3e0b-4d0e-9dd1-73fe4e35e1f7 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.282030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.283092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.283605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.284099 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.285053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:08.285613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.287247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0803529-f1e7-40ac-b8a4-0ad33c808520 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.296570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 279/559] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:08.298312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4a621-9c44-453c-927a-d10d9a24880e None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.298807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4a621-9c44-453c-927a-d10d9a24880e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.299322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4a621-9c44-453c-927a-d10d9a24880e None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.301029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 280/560] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:08.301960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4a621-9c44-453c-927a-d10d9a24880e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.366758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f080a4-08b3-4659-b56c-476a0a79554c None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.390499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.415051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.415834 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.416177 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.416515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.417365 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:08.417682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.419315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.432309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4a621-9c44-453c-927a-d10d9a24880e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.437807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 281/561] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:08.437807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.472008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92b2e3-e91e-4e7d-81fe-13026b565aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.472818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92b2e3-e91e-4e7d-81fe-13026b565aff None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.473096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92b2e3-e91e-4e7d-81fe-13026b565aff None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.473424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92b2e3-e91e-4e7d-81fe-13026b565aff None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.484020 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92b2e3-e91e-4e7d-81fe-13026b565aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-7b92b2e3-e91e-4e7d-81fe-13026b565aff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:08.491827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:08.492951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:08.493938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 281/562] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:08.495072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.495072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.495072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.513422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:08.514779 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:08.517034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.557599 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d75f8-7f5b-4f1d-bcb8-5c10035f1bce None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.587170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 282/563] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:08.588360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.594010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.594823 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.595097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.595362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.607832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.609388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:08.627182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:08.628297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:08.629294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:08.629294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:08.629294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:08.629294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:08.629294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 282/564] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:08.629294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.656884 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.698882 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6287c9-dbde-4224-a81e-ebc7a117587c None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.732484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8a31b-4332-4633-aa72-20829bcdd0a4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.733615 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.734341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.734555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.734830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.735651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=1ee839fa0ec84d8dbb939572f797bec2) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:08.735954 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.737414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8535dd-cbd9-4ab2-bfde-9e740f6a7199 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.758129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 283/565] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/role_assignments?user.id=1ee839fa0ec84d8dbb939572f797bec2&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:08.758129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d4b9e-2440-4ef4-ab03-f97594a76325 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.758129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d4b9e-2440-4ef4-ab03-f97594a76325 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.758129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d4b9e-2440-4ef4-ab03-f97594a76325 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.758129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 284/566] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:08.758129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d4b9e-2440-4ef4-ab03-f97594a76325 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.761846 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.776638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.777370 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.778030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.778030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.778664 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:08.778962 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.780377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.790194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 283/567] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:08.791207 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:08.882312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d4b9e-2440-4ef4-ab03-f97594a76325 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.897062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:08.910008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9f487-6ed3-467b-bad3-41178a1e3791 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:08.921279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208baa64-b477-48ff-8608-7f9f42f39b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.922108 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208baa64-b477-48ff-8608-7f9f42f39b21 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.922379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208baa64-b477-48ff-8608-7f9f42f39b21 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.922649 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208baa64-b477-48ff-8608-7f9f42f39b21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.935918 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208baa64-b477-48ff-8608-7f9f42f39b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-208baa64-b477-48ff-8608-7f9f42f39b21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:08.943179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:08.944295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:08.945304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 285/568] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:08.946150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.946270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.946563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.962333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:08.963016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:08.963300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:08.963576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:08.964159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:08.964361 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:08.964611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:08.965407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:08.966099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:08.980448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 284/569] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:08.981070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.032318 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.053280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 286/570] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:09.053280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.108288 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4278b-5160-4459-b501-a2fe321ed319 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.144653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.145489 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.145766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.146033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.155560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a106b8-b7ba-4b24-b5a2-37761efae37c None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.157052 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.158560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:09.176381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:09.177940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 285/571] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.183950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.207917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.208855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.209040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.209215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.210120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:09.210449 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.211869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.221331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 287/572] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:09.227469 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.290434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.298027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74611343-c1bb-4a66-aa35-a457d80a3db1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.334524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.335414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.335760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.336093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.343529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.343687 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf23f8-4d5e-459d-9c4c-56083a1737e2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.345106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:09.361781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:09.363107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:09.364383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 286/573] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:09.365079 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.383866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.384765 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.385121 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.385503 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.396819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:09.404983 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:09.406277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:09.407415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:09.412729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 288/574] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:09.413564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.436092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.480118 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225590d-2544-4a55-897f-0123b3009320 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.520333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.521013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.521362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.521600 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.522658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:09.522780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.524460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.529046 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa85ca-5392-47d2-9b9e-bdcf31d8af5b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.536805 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 287/575] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:09.537917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.549066 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.576311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.577789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.578370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.578942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.596570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:09.628543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:09.629733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:09.630880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:09.630880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:09.630880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:09.630880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:09.638409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 289/576] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:09.638409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.675035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef1c54-7554-40f3-9e64-c007c447f1d0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.700001 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.713569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.714248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.714525 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.714793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.715632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:09.715933 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.717415 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.734953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 288/577] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:09.735747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.746169 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adcd05-ebe1-460e-b3f5-89ae128f32e0 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.782756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.783762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.784258 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.784537 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.785415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:09.785733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.787114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.800809 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 290/578] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:09.800809 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.830556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.849445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeace76f-a2a6-487b-8abc-7374eb731c38 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.890721 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.891528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.891839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.892069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.900586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.902080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.909895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852152c-e6ca-4df8-aff6-5728f4099478 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:09.916754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:09.917891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 289/579] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:09.919793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:09.948761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:09.949451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:09.949731 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:09.950176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:09.950733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:09.951114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:09.952452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:09.965089 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 291/580] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:09.965469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:09.966776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.034735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd442cc4-c40a-4ed4-bd40-b61a02a71588 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.068887 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.080860 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.081704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.081987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.082236 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.082350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff966410-86b4-427a-b730-a10fa9e62387 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.093079 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.094618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:10.114183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:10.115322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:10.116542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:10.116542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:10.116542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:10.116542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:10.116542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 290/581] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:10.116542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.122627 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.123443 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.123716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.123986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.134533 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.135972 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:10.154263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:10.155233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:10.156247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:10.156247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:10.156247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:10.156247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:10.160270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 292/582] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:10.160964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.223328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.234254 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68611ec-0032-45fe-b58d-a4cd3c017bad None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.269781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4ce2-2967-4be3-ba9d-fcfc64db9cfc None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.270212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.270893 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.271174 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.271440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.272344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:10.272642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.274574 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.292984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 291/583] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:10.293924 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.311223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.312154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.312434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.312707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.320200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.321611 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.324162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:10.337274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:10.343357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 293/584] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:10.344566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.405706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e609-3d44-4cce-ac0e-287af2d81b0c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.440231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.441173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.441499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.441771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.442658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:10.443296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.444753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.446150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5901da1-5250-48e0-8859-36ef2ecf2851 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.456312 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 292/585] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:10.461637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.474212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.483435 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.484257 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.484655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.485018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.485877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:10.486330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.487761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.499474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 294/586] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:10.505038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.566461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.576197 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c3407-c7a1-452e-9ffe-c97e805af5b4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.613515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.614073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20292b98-9c89-4e00-8915-72052c47ade2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.614328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.614628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.614874 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.628515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.630478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:10.648745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:10.650336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:10.651506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:10.651506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:10.651506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:10.651506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:10.651506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 293/587] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:10.651506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.657182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.657963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.658238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.658564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.659395 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:10.659701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.661140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.679141 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 295/588] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:10.680302 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.729615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.772976 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518e9fb-6cb3-47bf-8189-0603df1acdaf None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.790381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbee13-5fd8-4e78-8e43-01aa149d59b6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.812730 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.815650 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.816476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.816759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.817034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.825273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.826809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.832460 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.833340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.833620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.833892 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.841439 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:10.841656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:10.842638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:10.848747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:10.849533 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:10.849634 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 294/589] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:10.849634 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:10.864277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:10.865653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:35:10.866940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:10.871292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 296/590] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:10.872083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:10.960670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f29f59-6e42-4c0e-a9a1-b81964d102c9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.983195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710771f1-33cb-43ec-b22d-c3535301fe37 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:10.983613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:10.996029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:10.996699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:10.996980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:10.997250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:10.998301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:10.998301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:10.999654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.011395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 295/591] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:11.012186 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.018739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.019548 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.019827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.020156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.031454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.032912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:11.050123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:11.051297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:11.052313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:11.052313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:11.052313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:11.052313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:11.052313 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 297/592] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:11.052638 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.081483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.115240 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07c50-fb63-46e4-8298-4fe7db6b8532 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.149827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.150499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.150781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.151057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.151846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:11.152151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.152473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7bd1e-a736-424d-930f-4b6019cfea7a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.153663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f3f45-5046-4bac-9cc7-d274a4dcfcc4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.165808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 296/593] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:11.167128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f33ac-b852-4ffb-8590-f256f67ac683 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.167436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f33ac-b852-4ffb-8590-f256f67ac683 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.167687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f33ac-b852-4ffb-8590-f256f67ac683 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.168907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 297/594] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:11.169624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f33ac-b852-4ffb-8590-f256f67ac683 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.193303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.194011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.194291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.194564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.195397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:11.195708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.197174 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.217934 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.218151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 298/595] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:11.218966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.280431 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f33ac-b852-4ffb-8590-f256f67ac683 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.311329 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.313616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.314430 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.314704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.314978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.321751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.323295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.324611 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4e34b-8205-44e8-8e60-99d09ac6ac4a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:11.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:11.338257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 298/596] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:11.339440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.363607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89325977-587e-4740-9d88-230dfe00596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.364288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89325977-587e-4740-9d88-230dfe00596e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.364570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89325977-587e-4740-9d88-230dfe00596e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.364852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89325977-587e-4740-9d88-230dfe00596e None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.365648 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89325977-587e-4740-9d88-230dfe00596e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:11.365989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89325977-587e-4740-9d88-230dfe00596e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.367378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89325977-587e-4740-9d88-230dfe00596e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.379382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 299/597] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:11.380696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.380995 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.381363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.398603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:11.400165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:11.442327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a45f2-d04b-4898-8064-a1a18f804ea0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.444105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.467179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 300/598] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:11.468091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.484434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.485814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.486097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.486374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/f5c6cd73a1234dde953e73fedc6440ee/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.502003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=f5c6cd73a1234dde953e73fedc6440ee, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.503482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.517366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 299/599] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:11 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/f5c6cd73a1234dde953e73fedc6440ee/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:11.518191 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.568393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.595453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df760c5d-d721-4a6d-a97e-c3505fd35676 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.637128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.637991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.638264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.638551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.639936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c91d6-a4d7-4a62-a01e-b5b7462a91f6 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.650760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.652245 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:11.671855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:11.673110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:11.674441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:11.674441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:11.674441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:11.674441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:11.674441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 301/600] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:11.674441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.688492 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.689333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.689688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.690030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.690905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:11.691318 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.692875 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.708029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 300/601] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:11.709945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.727390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.792933 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a200a-d255-4861-8616-e1906dc30837 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.818199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa0a5b-be64-4fc3-817c-a3c2e10bbfec None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.824523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.831908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.832720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.833001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.833303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.841006 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:11.842521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:11.856870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:11.858016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:11.859015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:11.859503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:11.859503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:11.859503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:11.859503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:11.862915 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 302/602] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:11.863763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.872797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:11.880810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:11.881958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:11.882922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 301/603] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:11.883730 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:11.959394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:11.976776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09b78-b93d-4ee6-b90c-83debdb4c392 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:11.993932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fb25e-8ea6-404b-bf9d-1240011f881b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.016473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.017432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.017739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.018399 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.019415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:12.019726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.022235 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.029654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.031508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.032462 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.032962 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/02409707922c4ddf9f8c896c13ef651a/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.035338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 303/604] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:12.036260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.051905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=02409707922c4ddf9f8c896c13ef651a, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.053458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.066690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 302/605] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:11 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/02409707922c4ddf9f8c896c13ef651a/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:12.067566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.114886 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:12.180105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3292-96a9-4392-8b0a-c8609666f7be None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.184934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff480e9-35f3-483d-b6d5-670194387ba4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.212709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:12.231968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.233374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.234275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.235020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.235990 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:12.236467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.238098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.239202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.240348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.240782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.241155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.242655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:12.243073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.244674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.250607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 303/606] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:12.251881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.265338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 304/607] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:12.266707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.395248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdc4c7-c36d-4b43-8e2d-30b00b4a1b27 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.395737 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75a2f1-7787-47c4-82a2-e6152ead90a7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.407575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:12.439305 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.440119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.440396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.440667 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.445253 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.446084 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.446372 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.446626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.453692 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.455188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.456195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.458579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:12.469945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:12.471048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:12.472144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:12.475910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 304/608] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:12.476749 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:12.482220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:12.483238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:12.484339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:12.484339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:12.484339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:12.484339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:12.484339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 305/609] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:12.484339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.540608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:12.600168 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273ce3f-5bb6-4051-8602-63bf408e9040 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.631996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962a84b-ac15-42c1-b45c-9c426eabf333 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.637257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.638286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.638664 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.639033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/c704d1a1b11e44a6889504246016996a/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.656287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=c704d1a1b11e44a6889504246016996a, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.657940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.685270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 305/610] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:12 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/c704d1a1b11e44a6889504246016996a/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:12.685270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.691527 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.692415 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.694146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.695645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.696619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:12.698253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.699886 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.728152 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 306/611] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:12.730626 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.787406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:12.913998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:12.924209 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506c29-2762-45dd-b80e-cc6e72433267 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.948843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c586a-5cca-4360-a889-d66cf01247a7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:12.969284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.970329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.970822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.971269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:12.972237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:12.972700 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:12.974328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:12.987740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 306/612] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:12.993487 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:12.994867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:12.996289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:12.997038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:12.997442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/bafd4f61582941a88f68e01deb85d4a7/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:13.017398 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=bafd4f61582941a88f68e01deb85d4a7, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:13.019498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:13.045380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 307/613] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:12 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/bafd4f61582941a88f68e01deb85d4a7/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:35:13.136832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d2ae-d066-40cb-8ff4-0960b3eccdb8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:13.191266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:13.192528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:13.192962 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:13.193437 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:13.204773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:13.206554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:13.227997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:13.229377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:13.230697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:13.232870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 307/614] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:13.238953 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:13.321854 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:13.363012 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c1e41-e91c-4650-af18-c4a7f9052a97 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:13.407795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:13.408750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:13.409211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:13.409647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:13.410671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:13.411128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:13.412798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:13.432767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 308/615] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:35:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:13.438045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:13.461359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:13.584115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45ed34-d0f3-410c-a90f-c0d76bcea2c5 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:13.632468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:13.633313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:13.633596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:13.633869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:13.653207 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:13.654744 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:13.676594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:13.677686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:13.679251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:13.679251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:13.679251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:13.679251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:13.679251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 308/616] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:13.683936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:13.835305 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293dd53-0b52-40cf-a1a4-a80851b2c13e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:13.865327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:13.878971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:13.879774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:13.880151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:13.880524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:13.881573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:13.881982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:13.883460 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:13.900445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 309/617] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:13.905642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:14.020902 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:14.068101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec27fb-9140-4e93-bde2-42d94434ce4f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:14.116951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:14.117823 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.118107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.118384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.126787 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:14.128319 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9b0a67af-95b3-4a47-8c32-20cafa898853 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:14.144396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:14.145495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:14.147180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 309/618] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:14.151877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:14.269480 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2580-9a50-4945-b428-4c75f45286c8 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.269870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa898e-feef-4b70-bd25-2694f4dca7f3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:14.270107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2580-9a50-4945-b428-4c75f45286c8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.270404 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2580-9a50-4945-b428-4c75f45286c8 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.272319 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 310/619] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:14.278015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.278423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.278822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.306470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:14.307414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.307846 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.308269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.309268 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:14.309744 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:14.311342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:14.318314 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:14.319587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:14.325082 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 310/620] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:14.335685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:14.397881 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:14.442999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efbac8-9705-4e3e-9313-94a660f60e94 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:14.482212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:14.483465 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.483975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.484504 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/1ee839fa0ec84d8dbb939572f797bec2/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.506589 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=1ee839fa0ec84d8dbb939572f797bec2, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:14.508378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:14.520414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 311/621] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:14.532621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 311/622] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:14 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/1ee839fa0ec84d8dbb939572f797bec2/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:35:14.545507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:14.636183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:14.702450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90458ca2-a152-4bff-925d-f09d980f97c0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:14.757140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ea2c2-863a-486d-8b69-3df3628fb11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:14.757997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ea2c2-863a-486d-8b69-3df3628fb11f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.758277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ea2c2-863a-486d-8b69-3df3628fb11f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.758550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ea2c2-863a-486d-8b69-3df3628fb11f None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.773560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ea2c2-863a-486d-8b69-3df3628fb11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-706ea2c2-863a-486d-8b69-3df3628fb11f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:14.782027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:14.783275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:14.784472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 312/623] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:14.791420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:14.841976 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28041557-ed75-4fa8-aaec-ed0876833c18 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.842481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28041557-ed75-4fa8-aaec-ed0876833c18 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.842859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28041557-ed75-4fa8-aaec-ed0876833c18 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.844607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 313/624] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:14.850100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.850497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.850894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.868985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:14.870434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:14.908563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7937d6-3a0f-4c3d-8f69-d5b3a2f3c4dd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:14.929650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:14.942765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 314/625] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:35:14.946811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:14.947792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:14.948353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:14.948933 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:14.949920 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:14.950412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:14.951972 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:14.965516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 312/626] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:14.966770 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:14.968085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.016020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:15.086157 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee665896-5abc-442a-9868-d13dc5e132cb None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.127192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed7d6f-1137-433f-aeb9-95e852d92a6f None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.159386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.160222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.160501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.160801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.176341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.177914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:15.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:15.204240 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:15.205283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:15.205283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:15.205283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:15.205283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:15.209436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.210525 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.211005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.211502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.218911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 313/627] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:15.224239 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.228083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:15.236910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:15.238017 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:15.238984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:15.241971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 315/628] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:15.248065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.267352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:15.400131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfac265-234d-4f4f-9184-738c89d3a8a2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.412617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:15.440722 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430640d9-0f4d-4067-96ef-2fb0c3af48e5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.477092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.478174 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.478657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.478899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.479391 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.479606 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.479888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.480162 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.480916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:15.481140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:15.481271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.482104 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.483142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:15.484101 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:15.497037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 314/629] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:15.500973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 316/630] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:15.507633 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.509599 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.599199 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:15.628868 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f193c1-7013-4ef9-a691-eecd2b924286 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.649607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb5d02-009c-4356-a6d1-ff0604a4f746 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.672897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.673863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.674137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.674414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.687111 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.688141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.688568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.688808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.689216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.690306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:15.698148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.699912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-83b1abed-9093-489c-9be9-e4ff4ca5ae77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:15.710278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:15.711440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:15.712733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:15.712733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:15.712733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:15.712733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:15.712733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 315/631] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:15.717280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60041f-9ba6-4fb1-af60-d6e9cc562342 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.717280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60041f-9ba6-4fb1-af60-d6e9cc562342 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.717280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60041f-9ba6-4fb1-af60-d6e9cc562342 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.717280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 316/632] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:15.718546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60041f-9ba6-4fb1-af60-d6e9cc562342 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-1fec9ba3-db76-496a-9887-600cb2539d77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:15.723248 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:15.724397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 317/633] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:15.725547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.726283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.726283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.744540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:15.745542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:15.769881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:15.817887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 318/634] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:15.819189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.847406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60041f-9ba6-4fb1-af60-d6e9cc562342 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.873194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:15.882643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.883461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.883748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.884022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.884756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:15.885065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.886475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988c6b1-f2b6-44f2-a7ca-b75d894530cb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:15.899602 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 317/635] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:15.900908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9b57a-af5b-443d-af7f-ce8b9e450b1c None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.901198 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9b57a-af5b-443d-af7f-ce8b9e450b1c None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.901500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9b57a-af5b-443d-af7f-ce8b9e450b1c None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.902864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 318/636] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:15.907525 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9b57a-af5b-443d-af7f-ce8b9e450b1c None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:15.930164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ba2b-1628-45a9-a56a-5769bea0ebe5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:15.969381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:15.970328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:15.970617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:15.971307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:15.972453 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:15.972766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:15.975377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.001493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 319/637] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:16.002748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.020132 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9b57a-af5b-443d-af7f-ce8b9e450b1c None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.020897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.056256 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dc1db-27f0-4fdc-9f8c-88de1a4cad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.057073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc1db-27f0-4fdc-9f8c-88de1a4cad87 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.057383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc1db-27f0-4fdc-9f8c-88de1a4cad87 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.057655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc1db-27f0-4fdc-9f8c-88de1a4cad87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.068668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dc1db-27f0-4fdc-9f8c-88de1a4cad87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-250dc1db-27f0-4fdc-9f8c-88de1a4cad87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:16.076403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:16.077720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:16.078692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 319/638] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:16.079472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.079472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.079472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.098383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:16.100370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:16.118281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c139c2-f6e2-431d-adbf-65fb0807740d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.144191 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.156643 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.157904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.158389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.158827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.167125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.168597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.176713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 320/639] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:16.176713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:16.195877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:16.197181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:35:16.198491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:16.200920 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 320/640] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:16.201958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.280835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.282459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23313e0b-f938-4e55-bd25-286063f6eb31 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.308946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6ec4e-186e-4dba-aa09-cc3b4dbc0015 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.317557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.318217 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.318494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.318793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.344629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.345501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.345943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.346223 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.346959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:16.347277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.348759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.359225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 321/641] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:16.360273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.368370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.394894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.396547 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.463112 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf64da9-9ee7-4279-89d8-5bb80bf2942a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.504794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.505774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.506085 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.506356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.512784 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 321/642] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:16.512784 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.513914 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.518530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:16.530369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:16.531670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:16.532802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:16.539126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 322/643] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:16.539126 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.604128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.628883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45193ede-2900-4915-b26d-d487b0cf870e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.637764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db8c7-1c10-4ded-bf6f-0e6fa78243e8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.664207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.665042 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.665443 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.665810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.666714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:16.667084 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.668527 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.675565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.676494 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.676727 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.676994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.690944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 322/644] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:16.690944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.692754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.694455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:16.712364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:16.713598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:16.714622 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:16.714622 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:16.714622 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:16.714622 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:16.714622 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 323/645] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:16.714931 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.772877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.803541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47718b-0c49-4bfd-a1be-0a2d73dc2dc7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.832566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d92f9-02b4-4ac2-9991-982ccf4adbcd None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:16.838963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.839776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.840058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.840322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.842322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:16.852540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.854805 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.871373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:16.872159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:16.872505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:16.872844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:16.873697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:16.874069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:16.875532 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:16.876993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:16.878190 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:16.884354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:16.884354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:16.884354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:16.884354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:16.884354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 323/646] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:16.884354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:16.897173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 324/647] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:16.897173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.000801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b646c6-0643-4b6c-8fee-2be77ebd3e2d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.013011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a419e62-51bc-473d-9f7f-044a8f80f600 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.013572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.040123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.040967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.041385 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.041769 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.042603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:17.042917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.044362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.049697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.050363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.050644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.050917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.062521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 325/648] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:17.062521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.109991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.123850 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.125399 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.162272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4315-03b1-49ed-8fe4-b72fd5debcf1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.197212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.197914 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.198196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.198478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.199210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:17.199516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.200921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.212211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 326/649] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:17.213011 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.224744 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 324/650] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:17.229440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.245569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.311500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.316741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125508-92a8-4c2a-88cb-de6e76df18bc None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.326798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b3436-717b-4d83-9be9-adc9531265d5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.351077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.351869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.352156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.352424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.360382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.361168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.361472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.361736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.362938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.364453 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.368994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.370455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c908ff4e-bb0e-459f-8160-ef4af61085fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:17.381476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:17.382703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:17.383719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:17.383719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:17.383719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:17.383719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:17.383719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 327/651] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:17.384036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd58611-3ae9-4822-870f-93873a80c5cb None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.384125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd58611-3ae9-4822-870f-93873a80c5cb None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.384410 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd58611-3ae9-4822-870f-93873a80c5cb None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:17.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:17.385704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:17.386781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:17.387530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 328/652] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:17.387530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd58611-3ae9-4822-870f-93873a80c5cb None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.389732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 325/653] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:17.390620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.473722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.489022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e57017-2ab0-4df8-bc2e-e6d063496315 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.491172 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd58611-3ae9-4822-870f-93873a80c5cb None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.528907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.529396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.529862 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.530148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.530335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.530463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.530609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.530876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.537151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.538709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.541161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.542622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.549414 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:17.552904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:17.554140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 329/654] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:17.555368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c3f534d0-e38a-4ea6-9793-9d86caceb0ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:17.562057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:17.563330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 326/655] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.565718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.582392 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:17.583493 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:17.651114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 327/656] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:17.651932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.661490 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4866e1-8e2c-4d32-86b6-fd418fe62f74 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.695007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.695664 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.695936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.696185 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.696904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:17.697201 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.698623 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.709709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 330/657] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:17.710491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.715273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.763304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14fe44-ef50-4ca2-866b-425cd345bff4 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.789356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:17.797926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.798612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.798895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.799172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.800008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:17.800319 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.801824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.810405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2c75c-16cd-4eb4-b41a-075d4e41e8e0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.819477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 328/658] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:17.820364 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.844593 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.845197 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.845510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.845786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.846583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:17.846894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.848350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.921681 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459019d-14aa-428c-bf51-d15aabdc5b68 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:17.960084 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:17.960771 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:17.961049 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:17.961353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:17.962087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:17.962390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:17.963794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:17.976416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 329/659] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:17.977283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:17.989280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 331/660] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:17.990155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.028243 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.057549 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.083359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc61d82-7c3d-49fc-b8c6-1ac8ff1a7ba9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.095311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04907-ed91-4891-ba50-d09038ecb0cb None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.116891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.117764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.118083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.118363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.128193 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.128989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.129106 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:18.129271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.129545 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.136570 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:18.137274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:18.138651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:18.139810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:18.140695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.140788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 330/661] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:18.140788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:18.155344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:18.156647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 332/662] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:18.162501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.240764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be1024-9da3-4f9b-b4c9-df7659116ba8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.247755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.260052 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fdf01-c10d-4ceb-8d73-68dc3c8e1733 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.275098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.275758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.276030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.276299 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.283549 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.294476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.295349 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.295702 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.296048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.302717 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.304215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:18.318873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:18.320065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:18.321075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 333/663] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:18.321676 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.351219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.353024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.426130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e45dc-40ad-415b-a962-7d852116c6e8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.453179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 331/664] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:18.454108 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.460398 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.461058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.461374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.461649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.462476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=f5c6cd73a1234dde953e73fedc6440ee) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:18.462788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.463562 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.464283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.470528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 334/665] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/role_assignments?user.id=f5c6cd73a1234dde953e73fedc6440ee&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:18.471226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.540182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.602975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61aae-9c76-415e-912e-8260e16bca1d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.607090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15770afa-ef32-4edf-bdeb-e44ac55d4a14 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.642971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.643651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.643927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.644195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.644946 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:18.645275 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.646799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.648740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.650472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.650759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.651039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.651918 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:18.652229 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.654286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.661315 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 332/666] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:18.661315 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.674563 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 335/667] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:18.675428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.734761 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.775656 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8745b-6126-420a-9e8c-d2d41932ffaf None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.792769 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef426f-c697-4814-9897-a87b0510a59c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.800401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.815606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.816351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.816561 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.816835 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.817787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:18.818094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.819659 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.829061 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.829903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.830183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.830458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.837908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 333/668] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:18.837908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.847062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.848651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:18.867560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:18.868655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:18.870296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:18.870296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:18.870296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:18.870296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:18.870296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 336/669] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:18.870296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:18.948683 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da1c54-e545-41fe-87b4-522b4517409b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.976175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:18.985708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:18.986518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:18.986804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:18.987081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:18.994695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507e99-605a-45f4-bd38-d8601db288ef None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:18.997942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:18.999409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:19.017606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:19.018711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:19.019728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:19.019728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:19.019728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:19.019728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:19.019728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 334/670] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:19.019728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.037897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.038564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.038846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.039119 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.072011 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.115704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.117190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.129866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2d082-9da5-4b6e-8b0e-a3aa098c9acf None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.168056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.168740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.169018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.169322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.170162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=02409707922c4ddf9f8c896c13ef651a) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:19.170471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.171938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.183053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 335/671] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/role_assignments?user.id=02409707922c4ddf9f8c896c13ef651a&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:19.183053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.233335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.245440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 337/672] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:19.246683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.299068 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e11c33-cc95-4225-bc64-f23e7b58480d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.329625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.337012 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.337847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.338193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.338534 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.339335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:19.339708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.341187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.354864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 336/673] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:19.355863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.387974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4459888-7810-47c0-8a90-bd592f1c4023 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.435362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.436025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.436299 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.436574 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.437373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:19.437638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.439040 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.450970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 338/674] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:19.451815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.470682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b55b-aa7c-43cd-a814-7be66e2ee0a5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.473616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.522817 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.524206 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.524811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.525344 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.537998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.540927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:19.567312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:19.568609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:19.570177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:19.570177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:19.570177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:19.570177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:19.572702 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 337/675] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:19.573714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.581424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a852-7508-4097-ad1a-e93b04c55e44 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.601714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.626342 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.627188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.627467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.627741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.636924 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.638682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:19.654845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:19.655910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:19.656894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:19.659485 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 339/676] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:19.660569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.696261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c498b03-9a42-4706-97fa-faba93c20683 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.745606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.746703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.746986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.747262 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.757762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.759925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.769513 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072dcb-52ff-4faa-b5ef-f32a90745dc3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:19.780296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:19.781347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:19.782405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 338/677] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:19.783052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.805161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.820180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.822054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.822503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.822931 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.823803 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:19.824401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.826786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.845291 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 340/678] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:19.845291 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:19.891122 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:19.916476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b4114-f04c-4678-9c25-641ef1c90b57 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.958829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:19.959654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:19.960004 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:19.960422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:19.961215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:19.961629 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:19.963401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:19.974170 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719101-3e4a-455b-a678-30d2699f42ff None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:19.980799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 339/679] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:19.981791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.017322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.019251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.019701 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.020144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.021129 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:20.021730 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.023344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.047826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 341/680] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:20.048911 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.070635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:20.096153 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb9959-a5ae-4338-8175-1b792514fb12 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:20.133811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.134560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.134842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.135128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.142895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.144288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.148241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:20.154576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9283e-7488-40b7-8717-9e17d29124b8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:20.161354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:20.162671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:20.163827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:20.167714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 340/681] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:20.169558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.194521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.195536 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.196005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.196452 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.197596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=c704d1a1b11e44a6889504246016996a) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:20.198107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.199733 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.206658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 342/682] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:20 2022] GET /identity/v3/role_assignments?user.id=c704d1a1b11e44a6889504246016996a&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:20.207697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.319356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e02e9-85d5-4c93-8009-736ed8e414b9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:20.338294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08be1b8-a028-4ab8-9ef4-3e0d8a35f6d0 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:20.358127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.359128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.359611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.360059 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.361048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:20.361563 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.363714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.377659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 343/683] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:20.383856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.402003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.402660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.402942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.403245 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.410098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:20.411932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:20.490191 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.491884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.495611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ac86d-2053-48e2-bd90-9cb3a0a72d41 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:20.545120 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.546295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.546816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.547305 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.548388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=bafd4f61582941a88f68e01deb85d4a7) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:20.548938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.550707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.558251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 344/684] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:20 2022] GET /identity/v3/role_assignments?user.id=bafd4f61582941a88f68e01deb85d4a7&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:20.614779 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 341/685] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:20.621336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.728148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:20.749799 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08c3f-89a3-4516-b9fa-4969aa310174 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:20.790997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:20.791824 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.792093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.792374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.808323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:20.809867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-40d955be-6eff-4e33-871c-fdd441ee11d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:20.827773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:20.829290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:20.830280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:20.830280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:20.830280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:20.830280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:20.830280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 345/686] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:20.835013 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:20.882715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:20.919141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff131ccc-1a12-4c24-a253-913b57b5b3e2 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.919865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff131ccc-1a12-4c24-a253-913b57b5b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.920355 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff131ccc-1a12-4c24-a253-913b57b5b3e2 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.922291 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 346/687] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:20.927118 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:20.927636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:20.928420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:20.948022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:20.949666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:20.957584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7861c4-d2e3-4c80-bd11-6714e4895a85 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.005043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.005716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.005820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.006874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:21.007184 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:21.008571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:21.026241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 347/688] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:35:21.033279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 342/689] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:21.033416 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.074686 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.133568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:21.204905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64b6eb-2073-4e5e-b62e-c81f64be2cb8 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.224226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:21.239658 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b03fe-b099-48f1-b885-2120b6a1d091 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.256050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.257157 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.257708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.258163 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.265814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:21.267572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:21.289849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:21.290987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:21.291997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:21.294725 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 348/690] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:21.300133 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.300974 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.301225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.301391 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.301565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.322817 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:21.341294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:21.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:21.343368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:21.345004 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 343/691] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:21.346260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.483283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63152f6-077f-453f-b769-3020eddf5b05 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.484286 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680f39-3e3d-40c9-a4b9-29d317e9b68c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.551502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:21.563272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.563976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.564260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.564540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.565322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:21.565633 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:21.567033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:21.577739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 344/692] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:21.577968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.578659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.578944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.579221 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.580024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:21.580329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:21.585758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:21.586992 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.595399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 349/693] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:21.602925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.615453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:21.755382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20d72e-e299-4df4-89aa-4f0c6029cd09 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.762493 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dacd5-356d-4616-8a1d-601251972f4f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:21.795087 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.796114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.796627 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.797124 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.798224 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=1ee839fa0ec84d8dbb939572f797bec2) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:21.798764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:21.799232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:21.800051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.800329 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.800603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.801432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d31869-094b-4a0d-86a0-2e4c97383c31 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:21.817274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 350/694] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/role_assignments?user.id=1ee839fa0ec84d8dbb939572f797bec2&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:21.817274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601e02d-cfa0-4390-9706-3a3e51b24f9f None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.817274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601e02d-cfa0-4390-9706-3a3e51b24f9f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.817274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601e02d-cfa0-4390-9706-3a3e51b24f9f None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.817730 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 351/695] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:21.819691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:21.819980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:21.820291 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:21.821408 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:21.823118 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:21.838255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:21.839526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:21.861274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:21.862353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:21.863374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:21.863374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:21.863374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:21.863374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:21.865999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 345/696] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:21.880101 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:21.902102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:21.929396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 352/697] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:35:21.975599 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:22.017668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:22.089209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5df8ba-7318-4bc6-af94-c6e22c36b5b8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.123515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f6f87-8be4-4e75-accb-1032f64a94cc None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.130605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.131702 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.132229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.132740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.133873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:22.134413 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.136554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.150263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 346/698] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:22.157127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:22.169559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.170750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.171251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.171745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.183692 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.185751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:22.206450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:22.207498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:22.208505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:22.208505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:22.208505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:22.208505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:22.211697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 353/699] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:22.220039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:22.255025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:22.284433 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473033c-4a89-4f2a-843c-9a080e27aa4e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.318539 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:22.325695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.326841 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.327419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.328002 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.334596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e12ad6-b9c3-432c-b10f-8586d140cc3e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.339579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.341311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:22.364657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:22.366078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:22.367162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:22.369337 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 347/700] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:35:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:22.372901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:22.381928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.382617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.382895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.383165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.384455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:22.384455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.386875 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.407057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 354/701] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:22.408995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:22.519106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:22.523795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0872-ade6-469f-b24e-1e4e77cb9284 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.529710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae5359-96f4-4f30-8db1-59bc4550a544 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.568513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.569654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.570735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.571252 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.572379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:22.572901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.574904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.597188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 348/702] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:35:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:22.606909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.607162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:22.607615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.608012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.608280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.609115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.610628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.640531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 355/703] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:35:22 2022] POST /identity/v3/users => generated 254 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:22.688599 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:22.758108 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6456-049f-4d24-9173-8ab3004dfa4b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:22.815107 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:22.815801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:22.816080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:22.816352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:22.913889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:22.915464 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:22.968629 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:23.027008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 349/704] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:35:23.028447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.141995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32248e1a-6ef3-4bd4-8e1c-b3ccd1d9c30a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:23.183588 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:23.184334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.184616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.184896 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.198914 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:23.200470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:23.229842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c5ac19a6-4129-40b1-aad3-8bd3c4f4e82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:23.233443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:23.234435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:23.235463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:23.235463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:23.235463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:23.235463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:23.235463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 356/705] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:23.241508 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.248002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2128-fe47-4417-ba66-15a790b4c832 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.248323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2128-fe47-4417-ba66-15a790b4c832 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.248546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2128-fe47-4417-ba66-15a790b4c832 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.250237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 357/706] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:23.256396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.256946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.257332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.277922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:23.279384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:23.379038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 358/707] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:23.408070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472b89-8e72-42e6-8695-4ad81c45ccf2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:23.444311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.446320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:23.447832 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.452437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.452724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.453506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:23.453815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:23.455215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6be57c-6e26-4669-abe8-8e9657a8e662 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:23.472156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 350/708] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:23.474330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650a88-1b18-4eb4-9a6a-efd9a0ef61b1 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.474689 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650a88-1b18-4eb4-9a6a-efd9a0ef61b1 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.474968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650a88-1b18-4eb4-9a6a-efd9a0ef61b1 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.476925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 351/709] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:23.481751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650a88-1b18-4eb4-9a6a-efd9a0ef61b1 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.493455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:23.575351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fff4-1f88-419f-8664-d9f69e78ee46 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:23.595577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650a88-1b18-4eb4-9a6a-efd9a0ef61b1 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:23.617604 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:23.630016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:23.631082 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.631359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.631620 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.632408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:23.634074 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:23.637082 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:23.638049 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.638383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.638715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.646577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:23.648137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-74c44aed-3316-4078-9f5a-7f2152b592a8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:23.650716 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:23.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:23.655199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:23.657183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 359/710] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:23.665261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.665261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.665261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:23.665830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:23.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 352/711] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:23.668598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.686518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:23.688123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:23.748126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:23.766949 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 360/712] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:23.767836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.788216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88330f-613a-49b9-b4f5-e5d6cd9c0607 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:23.832155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:23.833124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.833560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.833859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.834751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:23.835056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:23.836907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:23.842316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 353/713] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:23.843349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:23.885718 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54faca0-c373-4e89-865a-5394ff8562e5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:23.913708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:23.928117 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:23.929327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:23.929755 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:23.930169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:23.931239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:23.931698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:23.933395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:23.947866 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 361/714] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:23.949020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:24.058955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604b2-a00b-4776-ab75-a2940b8fe527 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:24.082739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb010e46-1ec0-472f-9e47-7d22ad20d18d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:24.096101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:24.097099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.097563 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.098011 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.098868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:24.099364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:24.101028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:24.107739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 362/715] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:24.108744 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:24.118648 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:24.132945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:24.133798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.134077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.134348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] PATH_INFO: `/v3/services/network` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.135011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:24.136416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:24.164756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:24.166214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:24.167296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:24.173195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 354/716] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:23 2022] GET /identity/v3/services/network => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:24.174383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:24.178883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:24.239033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f852-ba49-4ce2-ae5e-d713a0f859b9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:24.303064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:24.304096 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.304768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.306238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.307346 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=f5c6cd73a1234dde953e73fedc6440ee) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:24.308116 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:24.309863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:24.324626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 363/717] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:24 2022] GET /identity/v3/role_assignments?user.id=f5c6cd73a1234dde953e73fedc6440ee&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:24.374506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399e99c-e8ab-4d15-8ac0-54024f0c235b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:24.412627 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:24.413859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.414383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.414880 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.415812 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] RBAC: Adding query filter params (name=network) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:24.416401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:24.418059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b0c5b-e125-4ba3-9b56-afd0bfc851b4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:24.424014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 355/718] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:24.430233 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:24.542761 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a545fe3-e9a3-4008-900c-0fded41d30af None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:24.557343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:24.581261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:24.582028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.582307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.582576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.583290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] RBAC: Adding query filter params (type=network) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:24.583588 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:24.585062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3329525-2b06-448e-bf9a-4598338a66cc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:24.594869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 364/719] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:24.886409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9396b7-20e1-4490-9f46-d9add7cc59a3 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.887198 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9396b7-20e1-4490-9f46-d9add7cc59a3 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.887832 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9396b7-20e1-4490-9f46-d9add7cc59a3 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.889804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 356/720] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:24.896290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:24.896962 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:24.897526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:24.916657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:24.918223 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:24.997931 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 365/721] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:25.024341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9396b7-20e1-4490-9f46-d9add7cc59a3 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:25.106910 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:25.154451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9396b7-20e1-4490-9f46-d9add7cc59a3 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:25.196428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:25.197267 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.197567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.197840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.211848 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:25.219756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:25.220764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:25.221752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:25.224916 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 357/722] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:25.227515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:25.365836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb9d5-c165-4ea3-a489-743db9866542 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:25.375896 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:25.411127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:25.411745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.412062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.412342 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.413107 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:25.413445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:25.414848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395cac4-6008-40cb-bf96-8fb378758f08 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:25.429039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 366/723] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:25.434535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:25.525407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253d151-92f7-4ce5-9638-9a42eee02bc7 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.525702 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253d151-92f7-4ce5-9638-9a42eee02bc7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.525977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253d151-92f7-4ce5-9638-9a42eee02bc7 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.527366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 367/724] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:25.532635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.532926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.533206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.557440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:25.558654 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:25.604032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bc5d-46f7-4118-b2c7-d665ee008d93 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:25.629599 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:25.655545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:25.656379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.656659 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.656933 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.678978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:25.680391 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:25.706675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:25.707750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:25.708770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:25.708770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:25.708770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:25.708770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:25.710460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 368/725] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:35:25.710707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 358/726] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:25.718241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:25.733923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:25.853772 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732a6a6-ac5a-4b91-a607-b3e1d452dbdb None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:25.887981 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:25.892898 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:25.893774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.894053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.894329 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.897591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db703079-d93c-4b60-b728-f3a2c0adba46 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:25.925902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:25.941790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:25.942400 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:25.942679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:25.942952 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:25.943793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:25.944798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:25.945801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:25.946507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:25.946507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:25.946507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:25.948811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 359/727] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:25.957842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 369/728] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:25.958440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:25.962924 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:26.023201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:26.105031 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b1f02-4eea-439a-976c-df68da6fffd7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.141129 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.141911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.142131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.142409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.143181 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:26.143483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:26.145293 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:26.154669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 360/729] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:26.160589 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:26.219348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f131db-a3ea-4e63-870b-6b7e688383e4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.267901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d9b60-9ef8-44da-aa18-e21ea13ac248 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.269594 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:26.316026 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.316917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.317177 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.317474 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.319965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.321062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.321583 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.322083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.325567 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:26.327031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:26.342465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:26.343680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:26.345334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 370/730] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:26.352377 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:26.354175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:26.354309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:26.373090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:26.374218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:26.375216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:26.375216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:26.375216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:26.375216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:26.377691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 361/731] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:26.385903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:26.397051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:26.542210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f95c8a-a919-4b5d-9502-1e452f4c5f1d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.578843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c833e-edcc-4b93-9f03-e907c90131d4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.609139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.609855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.610135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.610411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.611256 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:26.611561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:26.613023 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:26.644766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.645473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.645756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.646055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.646776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:26.647076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:26.648461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:26.650943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 371/732] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:35:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:26.656220 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:26.673893 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 362/733] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:26.684644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:26.698564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:26.799095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:26.844285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bc573-3872-42eb-9cc9-21b3dbe3de90 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.872206 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a217ad-0e24-4a21-9cf3-c202574a87a5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:26.921132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e73a9a-c14e-46ff-91fb-72b9e4d920df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.921859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73a9a-c14e-46ff-91fb-72b9e4d920df None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.922141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73a9a-c14e-46ff-91fb-72b9e4d920df None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.922420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73a9a-c14e-46ff-91fb-72b9e4d920df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.974715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:26.975613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:26.975889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:26.976159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:26.994118 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:26.995742 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.002855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e73a9a-c14e-46ff-91fb-72b9e4d920df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.005113 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e73a9a-c14e-46ff-91fb-72b9e4d920df None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:27.025815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:27.026934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:27.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:27.033599 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 363/734] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:35:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:27.034505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.094395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.129901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 372/735] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:35:27.132140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309f808-c17f-427c-8b89-d99c1f647568 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.132900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309f808-c17f-427c-8b89-d99c1f647568 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.133925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309f808-c17f-427c-8b89-d99c1f647568 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.136292 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 373/736] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:27.137645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309f808-c17f-427c-8b89-d99c1f647568 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.146997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.188597 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a6121-ba34-420b-a257-4de3724b550d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.226551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.227241 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.227647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.227937 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.228884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:27.229193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.230661 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770725cc-6e16-43ec-9b8d-9f9ef0633bcf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.255893 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309f808-c17f-427c-8b89-d99c1f647568 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.256147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 364/737] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:27.256147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a1a2c-6bbe-4f26-8691-029febfb62e5 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.256147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a1a2c-6bbe-4f26-8691-029febfb62e5 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.256147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a1a2c-6bbe-4f26-8691-029febfb62e5 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.256147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 365/738] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:27.256985 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c790-d37f-4f5b-855f-016721048137 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.257432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c790-d37f-4f5b-855f-016721048137 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.257626 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c790-d37f-4f5b-855f-016721048137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.276789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-c7e8c790-d37f-4f5b-855f-016721048137 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:27.278019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-c7e8c790-d37f-4f5b-855f-016721048137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:27.302259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.303255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.303541 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.303818 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.314165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.315651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:27.334707 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:27.336382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:27.337526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:27.337526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:27.337526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:27.337526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:27.337526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 374/739] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:27.337526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.353855 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 366/740] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:27.355472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.355770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.356041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.373976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:27.375273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:27.382905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.427663 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.447478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 367/741] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:27.448622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.462635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de662e5-e40a-4e3e-9ad9-e22e7997ed07 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.499879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dddd9-3afb-49b8-8585-eac70c594f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.500540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dddd9-3afb-49b8-8585-eac70c594f5a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.500826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dddd9-3afb-49b8-8585-eac70c594f5a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.501106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dddd9-3afb-49b8-8585-eac70c594f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.551673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8b51-e82c-4c56-a81b-d8710d107bbd None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.568219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dddd9-3afb-49b8-8585-eac70c594f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.569747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dddd9-3afb-49b8-8585-eac70c594f5a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.584879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.585581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.585861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.586128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.586859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:27.587161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.588553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.598320 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.599823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 368/742] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:27.600733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.647146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.675819 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 375/743] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:35:27.677284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b64d39-8110-4920-aa40-19a14bbe952e None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.677709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b64d39-8110-4920-aa40-19a14bbe952e None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.677988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b64d39-8110-4920-aa40-19a14bbe952e None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.679489 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 376/744] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:27.680015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b64d39-8110-4920-aa40-19a14bbe952e None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.706162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f0e1-ad7d-427b-93f4-56e86ffe5aff None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.740076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.740940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.741209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.741512 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.751604 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:27.759249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:27.760310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 369/745] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:27.761287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.785954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b64d39-8110-4920-aa40-19a14bbe952e None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.811812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.820287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.820967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.821261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.821565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.822279 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.823704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.837655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 377/746] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:35:27 2022] POST /identity/v3/services => generated 251 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:27.838322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:27.875335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b53f-0652-4051-843e-b71b3625774c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.878501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:27.939940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:27.940684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.940960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.941229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:27.952617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:27.954177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-7f309ecf-470e-43d1-99e7-44deb65b67cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:27.975092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:27.976368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:27.977650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:27.981487 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 370/747] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:27.982131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3193df-60b7-4147-9ac7-1c556ad08f1e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:27.982952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:27.983233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:27.983498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.008119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:28.009400 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:28.020447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.021340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.021627 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.021898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.034117 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.035714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:28.054821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:28.055908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:28.056963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:28.056963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:28.056963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:28.056963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:28.056963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 378/748] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:28.057958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.093276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 371/749] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:35:28.093276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.102076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:28.166802 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf8779-0414-4691-8dd3-bcc31e6fa7b7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.197096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:28.202751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.203658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.203929 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.204208 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.204937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:28.205257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.206653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.208344 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce947bed-b9a4-4a3b-8f28-bf17051a7240 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.215312 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 379/750] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:28.216198 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.248016 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.248700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.248977 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.249266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.250480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:28.250795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.252217 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.264134 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 372/751] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:28.264929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.338660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c25461-6003-4f97-9d49-f5d47acd1411 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.343312 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:28.377733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf474e-9ff3-4720-9ccf-5cedfd25ad7f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.384084 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.384766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.385503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.385788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.386527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:28.386834 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.388639 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.407585 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 380/752] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:28.407585 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.420477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.421300 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.421593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.421867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.422601 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.423993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.439951 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 373/753] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:35:28 2022] POST /identity/v3/services => generated 240 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:28.441207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.466114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:28.541856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0cf5-5b77-4bc8-ac8d-fa88fbec9476 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.589825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.591047 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.591517 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.592066 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.604686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.606900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.607052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998ae99-3cff-4a43-afaa-57bfe6f64439 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:28.628460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:28.629569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:28.630557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:28.630557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:28.630557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:28.630557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:28.633063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 381/754] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:28.634121 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.649467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:28.658342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.658520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.658795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.659064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.660092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=1ee839fa0ec84d8dbb939572f797bec2) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:28.660204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.661716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.668009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 374/755] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/role_assignments?user.id=1ee839fa0ec84d8dbb939572f797bec2&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:28.668699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.770483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:28.788314 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4f237-835a-4f0f-9ebd-5cedaec1df8a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.820516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b897-3a76-4563-a833-dc658df81c99 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:28.838706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.839561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.839847 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.840124 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.856930 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.858514 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:28.887551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:28.888681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:28.889766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:28.901100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 382/756] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:28.902074 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:28.907617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:28.908312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:28.908595 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:28.908868 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:28.909631 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:28.909938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:28.911330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:28.931621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 375/757] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:28.947068 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:29.009275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:29.059086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec0ef8-8bb0-49cd-bf0b-5993418571ad None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:29.064238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cc0c1-9fa2-401b-a4e5-c85092204377 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:29.095419 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:29.096235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:29.096521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:29.096798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:29.097828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:29.098141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:29.099796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:29.110607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:29.113685 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 383/758] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:29.114091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:29.114462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:29.114740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] PATH_INFO: `/v3/users/nova` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:29.117227 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:29.119503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:29.135914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:29.137446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:29.164545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:29.166873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:29.169182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 376/759] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:35:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:29.179141 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:29.290074 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668af9-e95d-4384-92b5-d6737dec49bb None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:29.321203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:29.327625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:29.328619 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:29.329144 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:29.329694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:29.331535 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=45013317d2524d2a87ff2cc558b409fe) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:29.332102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:29.333864 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:29.340321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 384/760] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:29 2022] GET /identity/v3/role_assignments?user.id=45013317d2524d2a87ff2cc558b409fe&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:29.373932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f73c9-2ffa-4dbf-96c7-49da8abd726b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:29.424941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:29.426108 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:29.426739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:29.427262 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:29.428558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:29.429112 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:29.430944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:29.459537 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 377/761] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:35:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:29.465497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:29.626330 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad42844-5d77-4d84-9e57-a293aa0213e0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:29.679742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:29.680669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:29.680996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:29.681309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:29.697726 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:29.699216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:29.721413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:29.722554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:29.723592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:29.723592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:29.723592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:29.723592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:29.723592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 385/762] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:29.728684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:29.816891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:29.874863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081a50a-d6a6-42e4-9f1d-072b4baec524 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:29.920769 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:29.921480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:29.921757 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:29.922026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:29.922751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:29.923054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:29.924437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:29.936782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 378/763] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:29.946668 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:30.067841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:30.138843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e95ed-a3db-45b3-b68c-e83229aefde3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:30.175804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:30.177001 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:30.177572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:30.178048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:30.185555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:30.187313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:30.202180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:30.203215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:30.204237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:30.206592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 386/764] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:30.212339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:30.369489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010f07b-0da7-4b6f-a195-66a8213792b9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:30.386413 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:30.415232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:30.415904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:30.416182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:30.416455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:30.417259 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:30.417567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:30.419023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd39cb-84af-4261-a73c-dae858daf6de None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:30.438725 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 379/765] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:30.446353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:30.523582 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15b1e-8234-432c-8d47-ef3f5e3ad117 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:30.523748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15b1e-8234-432c-8d47-ef3f5e3ad117 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:30.524335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15b1e-8234-432c-8d47-ef3f5e3ad117 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:30.526060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 380/766] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:30.533634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:30.534314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:30.534751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:30.554178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:30.555387 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:30.615939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f69b-dab9-4cb0-8891-043eedacbb1b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:30.624356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:30.656892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:30.658378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:30.658871 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:30.659373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/f5c6cd73a1234dde953e73fedc6440ee/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:30.661647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=f5c6cd73a1234dde953e73fedc6440ee, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:30.663751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:30.679858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 387/767] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:30 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/f5c6cd73a1234dde953e73fedc6440ee/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:30.704272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 381/768] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:30.743621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:30.863028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66c571-b019-4356-acfd-88927e8c829e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:30.903673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:30.905012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:30.905610 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:30.906119 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:30.907012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:30.908664 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:30.922859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:30.923985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:30.924945 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:30.928361 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 388/769] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:35:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:30.934266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.058476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b674-181e-4c6d-bc39-0820259ccfda None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:31.059935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:31.102371 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:31.103060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.103475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.103752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.104459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:31.104760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:31.106157 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:31.117299 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 382/770] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:35:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:31.122932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.166642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:31.256338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d1cb-5d3b-497a-a3de-8809a0a7d37b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:31.293905 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:31.294986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.295502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.295980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.296903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:31.297521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:31.299231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf722f5-73c4-4f4e-9363-3d0dd68c194e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:31.305479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 389/771] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:35:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:31.314448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.464140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b0f3-48a6-40b6-9b89-8e4624e4ca4c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:31.477219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5373bf2-014f-4fd2-ac37-b19186ee5825 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.477753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5373bf2-014f-4fd2-ac37-b19186ee5825 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.477837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5373bf2-014f-4fd2-ac37-b19186ee5825 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.480648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 390/772] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:31.481839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804d3f-4b96-4ce7-8975-4e5f52b476a7 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.482116 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804d3f-4b96-4ce7-8975-4e5f52b476a7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.482393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804d3f-4b96-4ce7-8975-4e5f52b476a7 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.483613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 391/773] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:31.488703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.488989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.489306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.512559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:31.513259 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.513556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.513831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.514744 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] RBAC: Adding query filter params (interface=public, service_id=2d02877a86c74343af1e022479d73b92, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:31.514924 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:31.516310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9426bf-7d87-4bd8-8e39-68808ac88205 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:31.522968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:31.524956 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:31.525436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 383/774] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:35:31 2022] GET /identity/v3/endpoints?service_id=2d02877a86c74343af1e022479d73b92&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:31.526821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf425-732d-46c3-a092-987a486ef969 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.527112 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf425-732d-46c3-a092-987a486ef969 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.527389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daf425-732d-46c3-a092-987a486ef969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.545429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-07daf425-732d-46c3-a092-987a486ef969 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:31.546563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-07daf425-732d-46c3-a092-987a486ef969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:31.572231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:31.602323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 392/775] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:31.603593 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.629470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 384/776] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:31.630436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:31.659630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf425-732d-46c3-a092-987a486ef969 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.711333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0239-4430-4e5f-ac01-82b9816b85c1 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:31.748175 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:31.749059 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.749424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.749800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.750642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:31.752355 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:31.758432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 393/777] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:31 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:31.759467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.769114 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daf425-732d-46c3-a092-987a486ef969 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:31.809072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:31.810432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:31.810989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:31.811543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:31.824943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:31.833906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:31.835079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:31.836059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:31.839320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 385/778] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:31.845092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:31.871822 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:31.895939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:31.920530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a8098-7dec-4871-9e0e-a928e6023da1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:31.974915 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81f43-5299-465c-b3d2-16e5c2b241b7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.005863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.007182 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.007673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.008169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] PATH_INFO: `/v3/services/network` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.009078 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.012921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-b6f377f1-104d-4875-ab52-b82dc7a51e97 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:32.038086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:32.039617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:32.040558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:32.043379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 394/779] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:31 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:32.044982 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df94ca1-05f0-470c-a2e0-202426cf6426 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.045515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df94ca1-05f0-470c-a2e0-202426cf6426 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.046009 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df94ca1-05f0-470c-a2e0-202426cf6426 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.047881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 395/780] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:32.052869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df94ca1-05f0-470c-a2e0-202426cf6426 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.056818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.057805 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.058092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.058366 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.060080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:32.060393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.061931 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a7d7-0772-472e-affb-2dfa2408a78f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.073222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 386/781] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:32.074837 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.075182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.075540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.099572 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:32.100543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:32.173646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df94ca1-05f0-470c-a2e0-202426cf6426 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.182609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 387/782] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:32.183698 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.200039 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.210405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.211104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.211393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.212061 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.212788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] RBAC: Adding query filter params (name=network) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:32.213098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.214535 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.219843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 396/783] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:32.220766 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.240086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.303048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa23bd-7cb1-43f8-a342-92ecdc6fda2a None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.323740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea56f1-d2ba-4ffc-942c-dc7f05f66eb1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.341834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.343033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.343565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.344055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.355960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.357740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.359067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.359877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.360158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.360449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.370957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:32.378058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:32.379033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:32.380070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:32.380070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:32.380070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:32.380070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:32.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:32.381351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 397/784] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:32.382382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.385844 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 388/785] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:32.386940 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.461198 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.461718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.484765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2154b4-5917-4c6d-ab04-81fd89c82a30 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.506306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771f70-ec26-46ca-a7aa-8b00aba4adb2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.522313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.523004 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.523282 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.523555 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.524840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] RBAC: Adding query filter params (type=network) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:32.524967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.526565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.531901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 398/786] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:32.532877 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.552971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.554079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.554566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.555075 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.556144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:32.556656 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.558304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.568591 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 389/787] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:32.569861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.635560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624ceb-abc2-46a0-9a6d-48fde1325791 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.670880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.671783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.671924 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.672132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.672870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:32.673206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.674594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.678992 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b4e65-194c-4094-b92a-b3764e6a2f10 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.683898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.687851 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 399/788] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:32.689206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.715699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.722688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.724075 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.724705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.725295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.728361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] RBAC: Adding query filter params (interface=public, service_id=0cd084ed278542699c975b49cddec089, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:32.728923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.730579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.738732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 390/789] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/endpoints?service_id=0cd084ed278542699c975b49cddec089&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:32.740346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.800605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee956e0-5af8-443b-be8c-9cbdd663a051 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.848607 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.850528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.851298 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.851779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.864756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.866284 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053b7b0-829b-4c08-abae-1c8fb5923b6f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:32.866431 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:32.885553 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:32.887526 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:32.888762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:32.888762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:32.888762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:32.888762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:32.888762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 400/790] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:32.888762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:32.916010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:32.917730 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:32.918017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:32.918323 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:32.930554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:32.932023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:32.956841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:32.957456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:32.958501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:32.959697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:32.963052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 391/791] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:32.964092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.019352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2409d02-72bc-4d80-aa2d-7401a69148e7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.022898 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:33.064283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.065143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.065550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.065939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.066913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.068551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.075368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 401/792] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:33.076457 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.098212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01c9c1-cb37-4f64-9a42-7f4ad315d935 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.135863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.136535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.136811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.137077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.137913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:33.138339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.139559 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.163748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 392/793] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:33.168920 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.218856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5091-caf9-47ab-9d3d-2ee5c97615b8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.240777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:33.263310 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.264311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.264589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.264860 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.265770 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:33.266078 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.267591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445351e0-5812-4fab-b793-21e980c3aebd None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.290274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 402/794] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:33.292397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7701-0b61-45ae-a3f5-837737705551 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.292782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7701-0b61-45ae-a3f5-837737705551 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.293181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7701-0b61-45ae-a3f5-837737705551 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.294794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 403/795] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:33.295457 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7701-0b61-45ae-a3f5-837737705551 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.367110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3f6ec-a62c-45dc-b828-0df73ca4d902 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.381636 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:33.421553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.422803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.423448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.423831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.428811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7701-0b61-45ae-a3f5-837737705551 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.433874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.435461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-6755f250-dff3-4157-9530-d8f3bef93503 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:33.457577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:33.458780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:33.459847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 393/796] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:33.462565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.462846 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.463116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.471932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.472748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.473018 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.473319 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.484559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.486083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.489738 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:33.491257 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:33.507789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:33.509045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:33.510337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:33.510337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:33.510337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:33.510337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:33.513354 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 404/797] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:33.514227 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.544323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:33.580956 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 394/798] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:33.582626 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.620143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275bc0f-9738-4075-a477-58434736a18d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.656869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.657590 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.657871 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.658143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.658988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:33.659291 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.660865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.683941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 405/799] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:33.683941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.710620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:33.714861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6deed9f-4d7d-4e3b-93d2-3f3b9182d174 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.753641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.754482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.754919 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.755293 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.756219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:33.756629 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.758180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.770235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:33.772759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 395/800] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:33.773745 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.792729 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848406d7-4bc8-488f-ae84-95997fe1b562 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.829712 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.830638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.830928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.831206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.842611 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:33.850955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:33.852178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:33.853155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 406/801] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:33.853947 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.887489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138e12d-9737-4c15-b635-6baf5f21d78e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.923976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:33.924824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:33.925104 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:33.925400 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:33.936556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:33.937991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:33.955764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:33.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:33.958203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:33.958203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:33.958203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:33.958203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:33.961927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 396/802] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:33.963003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:33.967212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300977e-9b9c-4c84-a177-7c7a0a5a43d8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:33.980375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.005980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.007216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.007783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.008269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.016208 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.018326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:34.034601 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:34.035635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:34.036791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:34.045146 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 407/803] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:34.046504 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.053665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.074766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3777da-632b-4fbc-b959-0eee9b00c3d0 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.111523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.112236 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.112519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.112795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.113729 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:34.113876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.115294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.129282 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 397/804] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:34.130200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.171852 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f79d-9b04-4146-a020-9745a44d91c8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.219244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.220420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.220708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.222776 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.224678 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:34.226225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.227719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.232550 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.247585 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 408/805] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:34.248655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.263426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeccea-74ea-4bbe-b2ee-93c80fdb2006 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.298442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.299145 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.299411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.299683 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.300467 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:34.300772 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.302230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.315187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 398/806] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:34.316005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.324978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.367588 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87265-beb2-4574-bde8-5fd4e7466ed4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.406211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.407028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.407306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.407578 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.422617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.423874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2913519-7c4c-4cc6-b2a3-720f72d579f9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.424078 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-ba3607ad-8d19-4bd6-bdfb-c12bd0e798c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:34.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:34.442311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:34.443382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:34.443382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:34.443382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:34.443382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:34.443382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 409/807] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:34.444142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ebd36-a03f-474b-8b08-51b0a097f8aa None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.444142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ebd36-a03f-474b-8b08-51b0a097f8aa None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.444399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ebd36-a03f-474b-8b08-51b0a097f8aa None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.445849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 410/808] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:34.450121 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ebd36-a03f-474b-8b08-51b0a097f8aa None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.466599 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.467728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.468037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.468361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.470802 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.477034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.478808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:34.493304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:34.494321 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 399/809] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:34.495312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.575484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ebd36-a03f-474b-8b08-51b0a097f8aa None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.593273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.607555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bb426-85e0-467c-9692-b2cad0ca0868 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.614463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685c69d-306d-48ee-818e-79b2e3d08986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.615836 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685c69d-306d-48ee-818e-79b2e3d08986 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.616120 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685c69d-306d-48ee-818e-79b2e3d08986 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.616393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685c69d-306d-48ee-818e-79b2e3d08986 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/1ee839fa0ec84d8dbb939572f797bec2/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.619052 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685c69d-306d-48ee-818e-79b2e3d08986 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=1ee839fa0ec84d8dbb939572f797bec2, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.620503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685c69d-306d-48ee-818e-79b2e3d08986 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.635714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 411/810] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:34 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/1ee839fa0ec84d8dbb939572f797bec2/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:34.637199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c6575-54db-496d-9973-0a353eb98412 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.637757 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c6575-54db-496d-9973-0a353eb98412 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.638223 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c6575-54db-496d-9973-0a353eb98412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.654134 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.655006 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.655359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.655705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.656570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:34.656947 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.658684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.659308 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-151c6575-54db-496d-9973-0a353eb98412 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:34.660840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-151c6575-54db-496d-9973-0a353eb98412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:34.677912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 400/811] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:34.677912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.740991 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.756924 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 412/812] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:34.758592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c6575-54db-496d-9973-0a353eb98412 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.795279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a2889-af2a-4559-aa3c-22c4f1b1f234 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.834947 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.835767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.836138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.836480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.837405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:34.837783 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.839743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.853423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 401/813] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:34.858125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:34.904532 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c6575-54db-496d-9973-0a353eb98412 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.915216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:34.955190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:34.956662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:34.957789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:34.958348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] PATH_INFO: `/v3/users/nova` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:34.970970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:34.973143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:34.991666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:34.992831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:35:34.994104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:34.994771 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c029106-994f-4c3e-88e5-ec953a799e03 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:34.996523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 413/814] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:35.002003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.010501 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.035285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.036276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.036701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.037082 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.037889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.039429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:35.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:35.053379 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:35.054343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 402/815] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:35:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:35.055296 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.125307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840eae-1c5e-4e29-81ae-add2dbbe022d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:35.160310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b32302-f946-42cf-9b74-da33981d8194 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:35.164130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.165693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.166173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.166686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/45013317d2524d2a87ff2cc558b409fe/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.185331 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=45013317d2524d2a87ff2cc558b409fe, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.187361 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.200636 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.201478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.201810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.202156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.203188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:35.203607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.205202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.205426 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 414/816] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:34 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/45013317d2524d2a87ff2cc558b409fe/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:35.206417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.229954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.257367 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 403/817] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:35:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:35.258622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.343922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c531e-695d-4dfe-a219-8f9885e2f873 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:35.374857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.385394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.386438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.386912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.387503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.388548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:35.389067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.391701 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.398063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 415/818] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:35:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:35.408660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.473516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111db7c-f37a-4bf8-bf35-f87b5d68ad73 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:35.522506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31a00c-c572-4530-b96c-fac4f04ef07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.523180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31a00c-c572-4530-b96c-fac4f04ef07d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.523462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31a00c-c572-4530-b96c-fac4f04ef07d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.523740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31a00c-c572-4530-b96c-fac4f04ef07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.549965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.575161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4d90d-1eab-49e2-bd7d-df3e4344b18e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:35.601347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31a00c-c572-4530-b96c-fac4f04ef07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.603023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31a00c-c572-4530-b96c-fac4f04ef07d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.619863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.620824 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.621103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.621407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.622365 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:35.622672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.624422 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6d8d8-ff0e-4aa7-a0a4-8cfc3f9e16e6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.631466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 416/819] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:35:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:35.633364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba69ff-3739-40f4-a3eb-e57c3b77e765 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.633649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba69ff-3739-40f4-a3eb-e57c3b77e765 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.633921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba69ff-3739-40f4-a3eb-e57c3b77e765 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.636189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 417/820] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:35.637639 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba69ff-3739-40f4-a3eb-e57c3b77e765 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.647460 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.723708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 404/821] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:35:35.725186 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.725497 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.725775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.742942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:35.744140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:35.770125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba69ff-3739-40f4-a3eb-e57c3b77e765 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:35.815798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:35.816472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:35.816789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:35.817053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:35.817851 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:35.819529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:35.830793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 405/822] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:35.831546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.853072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.859881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 418/823] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:35:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:35.860982 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:35.933428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:35.984222 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848f10a-bff1-4c5c-883e-53aad224d870 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.010281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febc92d-76ed-42d7-a9c7-f01ca2c5a730 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.025577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.026681 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.027120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.028216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.041749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.043646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:36.065606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:36.066679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:36.067677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:36.067677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:36.067677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:36.067677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:36.071002 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 406/824] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:36.077406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.099988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.101211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.101696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.102136 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] PATH_INFO: `/v3/services/network` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.102977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.106188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:36.124321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:36.125783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:35:36.127039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:36.136266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 419/825] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:35 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:36.143986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.242815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e619c-8fe6-4025-803a-93dbac811cf4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.243720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:36.263363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:36.283346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.284446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.284989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.285515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.286475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:36.287183 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.288856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.293912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f03eb-b6e0-499e-ba8a-481a160435f0 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.303057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 407/826] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:36.311004 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.348158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.350087 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.350755 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.351352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.352508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] RBAC: Adding query filter params (name=network) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:36.353400 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.357166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.368018 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 420/827] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:36.373975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.443268 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a38e58-3b0f-46df-99a0-11957857db1c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.483737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.484970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.485566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.486055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.493764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.495902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:36.515718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:36.516825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:36.517870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:36.521516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 408/828] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:36.529905 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.607719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:36.609359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43389dc3-0366-4dab-a4cf-85e6f9e02950 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.657721 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:36.663041 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.663736 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.664019 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.664295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.665014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] RBAC: Adding query filter params (type=network) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:36.666109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.667533 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.680030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 421/829] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:36.684560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.736052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44b13a-eb7e-4fbb-937e-61a822f81345 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.799401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.800080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.800360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.800637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.801459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:36.801771 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.803423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.828084 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 409/830] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:36.834076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:36.859869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21488400-58be-4329-91c2-a5cedf0804e9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:36.923044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3ef5-c3ac-458e-977c-b1c0992bd5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:36.923728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3ef5-c3ac-458e-977c-b1c0992bd5df None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:36.924008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3ef5-c3ac-458e-977c-b1c0992bd5df None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:36.924279 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3ef5-c3ac-458e-977c-b1c0992bd5df None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:36.926166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a3ef5-c3ac-458e-977c-b1c0992bd5df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:36.927625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a3ef5-c3ac-458e-977c-b1c0992bd5df None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:36.965739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 422/831] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:35:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:37.011182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:37.016083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f23b-423e-45cc-b12d-3a8c533ec64a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:37.072081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:37.072789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.073073 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.073422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.074337 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=f5c6cd73a1234dde953e73fedc6440ee) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:37.074653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:37.076188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df11ba-8f6e-47d5-9803-534d5d640716 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:37.086955 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 410/832] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:36 2022] GET /identity/v3/role_assignments?user.id=f5c6cd73a1234dde953e73fedc6440ee&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:37.255068 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb2ae3-69de-4a5f-a42f-276d988a4adf None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.255281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb2ae3-69de-4a5f-a42f-276d988a4adf None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.255727 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb2ae3-69de-4a5f-a42f-276d988a4adf None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.257071 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 423/833] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:37.263221 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.263564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.263792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.285054 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:37.286399 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:37.369342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 411/834] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:37.393937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb2ae3-69de-4a5f-a42f-276d988a4adf None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:37.451371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:37.519665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb2ae3-69de-4a5f-a42f-276d988a4adf None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:37.558110 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b8cbd-2d50-48c0-8985-5f5958451341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:37.559387 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b8cbd-2d50-48c0-8985-5f5958451341 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.559882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b8cbd-2d50-48c0-8985-5f5958451341 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.560381 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b8cbd-2d50-48c0-8985-5f5958451341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.576773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b8cbd-2d50-48c0-8985-5f5958451341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-d48b8cbd-2d50-48c0-8985-5f5958451341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:37.586579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:37.587734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:35:37.588889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:37.597353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 424/835] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:37.602455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:37.698346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:37.711488 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0fe7d-a2b0-41da-a4e6-d21ad9b393bc None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:37.747848 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:37.748654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.748830 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.749084 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.749962 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:37.750277 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:37.751674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:37.756433 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc0e7-b437-4a4e-a1e8-7669cf1e130d None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.757396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc0e7-b437-4a4e-a1e8-7669cf1e130d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.757898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc0e7-b437-4a4e-a1e8-7669cf1e130d None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.759793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 425/836] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:37.763341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 412/837] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:37.766647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.766932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.767045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:37.767383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.787109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:37.788717 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:37.861148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 426/838] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:37.872471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f4c59-2899-4c58-8bcd-73e3cfd3a04e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:37.884173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:37.908141 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:37.909532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:37.909865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:37.910352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:37.923139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:37.925471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:37.929685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:37.952001 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:37.953175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:37.954294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:37.954294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:37.954294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:37.954294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:37.955351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 413/839] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:37.965281 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:37.965507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.025091 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75804928-cc6e-409a-bb5e-a0161db74bb4 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.070656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.072038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.072954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.073449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.087909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:38.093575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491b990-edae-4b75-b20f-5a0fc4657925 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:38.098653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:38.099948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:35:38.101094 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:38.108439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 427/840] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:38.109281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.141677 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.142360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.142638 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.142938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.143907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:38.144218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.146317 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.165331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 414/841] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:38.169967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.237994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.248349 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59f50-368f-463b-8e03-68ab7511c8fa None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.251465 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.294940 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.295798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.296087 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.296363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.297119 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:38.297455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.299673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.302518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d621aae-92b7-488d-86cc-2fec943e862e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.312305 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 428/842] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:38.318846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.344745 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.345605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.345887 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.346161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.354778 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.356359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:38.376212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:38.377390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:38.383420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 415/843] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:38.386106 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.446138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43531148-b0d1-46eb-8a07-457993bd0473 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.494687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.495750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.496036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.496695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.512393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13c6f8-d85a-4d14-8b72-c3840371fbb3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.516786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.518921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.541251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:38.553099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:38.554211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:38.555259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:38.555259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:38.555259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:38.555259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:38.557546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 429/844] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:38.558419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.559320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.559807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.560284 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.561354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:38.561836 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.563051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.564840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.579170 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.587444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 416/845] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:38.592504 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.692327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc794de-a016-400a-8b19-1d927fc9e31e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.709712 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4042228-3c1d-475f-b1db-9f580eb5ab66 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.730708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.731731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.732215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.732722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.733615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:38.733928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.735450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259be166-81b9-49b4-b6e7-dc809b14e67f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.745894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.746660 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.747007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.747350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.754070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 430/846] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:38.755708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb262-fb0c-4d2a-806b-564a46a24084 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.755988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb262-fb0c-4d2a-806b-564a46a24084 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.756687 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb262-fb0c-4d2a-806b-564a46a24084 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.757970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 431/847] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:38.758509 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcb262-fb0c-4d2a-806b-564a46a24084 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.807043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.828196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.829804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.834749 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:38.870829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcb262-fb0c-4d2a-806b-564a46a24084 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:38.907743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:38.908570 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.908851 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.909844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.917077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:38.918576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-cdaae717-48fb-4008-b049-dfb493ca69a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:38.934804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:38.941407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 432/848] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:38.942656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:38.943560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:38.944957 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 417/849] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:35:38.946704 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:38.966898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:38.968123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:39.035187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.049933 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 433/850] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:39.050816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.068078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0f16f-6f7a-4ffb-b3f8-c44e554e062a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.106715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.108043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.108510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.108975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.110077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:39.110607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.112214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.126405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.131278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 418/851] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:35:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:39.131278 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.177990 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ea66-3369-4a3b-9687-8c87f0f216e0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.215491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.216313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.216595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.216868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.230310 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.231857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.244582 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2844e5-24c9-4801-9e0a-be53dacd8325 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:39.250919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:39.251948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:39.252946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:39.252946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:39.252946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:39.252946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:39.252946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 434/852] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:39.254263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.285080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.285949 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.286228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.286505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.287292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.288670 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:39.300642 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:39.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:39.302879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 419/853] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:39.303775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.306874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.373548 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b882366-1c4f-4c34-991c-dd5f08f2b2c3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.391287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.418796 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617f4d9-d371-4f8b-9723-23620259970f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.419447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.420333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.420974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.422363 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.459232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.459931 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.460207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.460485 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.461217 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:39.461548 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.462988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.475774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 420/854] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:39.476730 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.516579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.518080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.549406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.595003 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb30ac9-ffb9-48ed-8786-61959de31581 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.626020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 435/855] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:35:39.626930 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.631109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.631787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.632060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.632327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.633029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:39.633368 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.634740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c1281-7d01-4dec-bac4-2346c0900153 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.639226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.639355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 421/856] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:39.640579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33006-07d7-4ee7-b4a5-ea0b90d8723e None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.640861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33006-07d7-4ee7-b4a5-ea0b90d8723e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.641149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf33006-07d7-4ee7-b4a5-ea0b90d8723e None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.642861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 422/857] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:39.643068 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33006-07d7-4ee7-b4a5-ea0b90d8723e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.728191 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8ea4-c624-49ad-a072-f6130631cd6b None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.743228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf33006-07d7-4ee7-b4a5-ea0b90d8723e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.763477 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.764277 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.764561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.764837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.768094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.771585 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.773057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.778132 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.778967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.779247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.779519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:39.787053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:39.788097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:39.789100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 436/858] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:39.789698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.793134 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.794621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-01542c3d-e832-4d48-a786-925fd4817472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:39.813116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:39.814760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:39.815852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:39.815852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:39.815852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:39.815852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:39.819150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 423/859] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:39.820438 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.820727 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.820999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.837771 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:39.838966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:39.859756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:39.893301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b69f7-5ed2-463e-9f8d-7f1795f428fe None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:39.909033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 424/860] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:39.909917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:39.933033 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:39.933753 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:39.934031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:39.934309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:39.935059 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:39.935376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:39.936770 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:39.941944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 437/861] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:39.949334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.035008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c961c4e-825c-4e0c-b291-356200ac8e9d None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.046265 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:40.077109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.077836 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.078103 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.078369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.079667 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.079982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.081474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.099185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 425/862] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:40.099371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80263e-fb74-4c42-a723-ef4b1383b062 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.099464 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.135850 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.136535 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.136822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.137090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.137987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.138364 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.139955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.141591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:40.154265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 438/863] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:40.158216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.207115 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a57ef7-dc87-42ae-9da9-e02657932979 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.244684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.245541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.245935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.246210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.246989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.248376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.260799 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972a3d-44a4-45b2-9c3a-62e2ae610655 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:40.261870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:40.262646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:40.263441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 426/864] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:40.264144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.297890 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:40.303058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.303911 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.304193 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.304470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.305357 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=1ee839fa0ec84d8dbb939572f797bec2) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.305672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.307138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.313323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 439/865] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/role_assignments?user.id=1ee839fa0ec84d8dbb939572f797bec2&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:40.322371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.382336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ac7f6-9e27-4de9-a82f-fe094902f27d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.417800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:40.423206 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.424349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.424841 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.425401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.434058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.435888 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:40.452380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:40.453201 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:40.454037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:40.457514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 427/866] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:40.462279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.476062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb48d35-1d5f-4a99-8cc0-2338afc00932 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.524929 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.526443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.526443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.526443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.526860 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.527160 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.528538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.533569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 440/867] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:40.540024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.572078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:40.582075 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fbe5-a453-484b-8b87-e4bfb0f1e9cd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.625509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.626454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.626731 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.626999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.627788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.628090 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.630213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.645427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 428/868] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:40.651837 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:40.676848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9120d-0107-4a70-9122-7abc1c3480ed None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.711977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:40.729595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.730284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.730569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.730844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.731559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.731863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.733827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54716-e11a-43d4-9fcb-39741d2348d3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.740914 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 441/869] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:40.768083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4df79-ee72-43e9-9bf1-4ad0b7dd300e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:40.811424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:40.812119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:40.812446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:40.812725 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:40.813641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=45013317d2524d2a87ff2cc558b409fe) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:40.814173 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:40.815428 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9f15-3887-4699-b297-67cff1b8d313 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:40.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 429/870] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:40 2022] GET /identity/v3/role_assignments?user.id=45013317d2524d2a87ff2cc558b409fe&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:42.221429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cedc05-330d-40eb-b44b-5d4a9e9142f1 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:42.221935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cedc05-330d-40eb-b44b-5d4a9e9142f1 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:42.222006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cedc05-330d-40eb-b44b-5d4a9e9142f1 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:42.223583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 442/871] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:42.229247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275c4da-6ade-48b4-8f16-c6dcbcc64961 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:42.229639 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275c4da-6ade-48b4-8f16-c6dcbcc64961 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:42.229820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275c4da-6ade-48b4-8f16-c6dcbcc64961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:42.247311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-e275c4da-6ade-48b4-8f16-c6dcbcc64961 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:42.248505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-e275c4da-6ade-48b4-8f16-c6dcbcc64961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:42.260076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:42.320863 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 430/872] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:42.343576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cedc05-330d-40eb-b44b-5d4a9e9142f1 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:42.455077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cedc05-330d-40eb-b44b-5d4a9e9142f1 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:42.491580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a0d38-b462-478c-a6cb-b497abefdd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:42.492166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0d38-b462-478c-a6cb-b497abefdd6b None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:42.492450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0d38-b462-478c-a6cb-b497abefdd6b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:42.492831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a0d38-b462-478c-a6cb-b497abefdd6b None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:42.493713 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a0d38-b462-478c-a6cb-b497abefdd6b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:42.495131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a0d38-b462-478c-a6cb-b497abefdd6b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:42.512643 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 443/873] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:35:42 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:42.718546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0645b6-07b1-46af-a854-e1cdbecacdfc None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:42.719213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0645b6-07b1-46af-a854-e1cdbecacdfc None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:42.719682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0645b6-07b1-46af-a854-e1cdbecacdfc None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:42.721567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 431/874] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:42.726739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:42.727238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:42.727703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:42.747148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:42.748705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:42.797764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:42.831779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 444/875] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:42.854169 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0645b6-07b1-46af-a854-e1cdbecacdfc None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:42.974133 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0645b6-07b1-46af-a854-e1cdbecacdfc None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.012661 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.013434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.013715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.013984 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.014644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:43.016029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-fdabb700-dc95-482f-b87d-0c769f5d0a62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:43.028971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:43.029874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:43.030600 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:43.031228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 432/876] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:43.036964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:43.070759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa33856-fef9-422b-bde1-a790495bc9ac None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.071055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa33856-fef9-422b-bde1-a790495bc9ac None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.071308 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa33856-fef9-422b-bde1-a790495bc9ac None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.072996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 433/877] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:43.078326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb8a8-ea50-4047-b19e-c57a4785ab5c None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.078615 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb8a8-ea50-4047-b19e-c57a4785ab5c None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.078897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb8a8-ea50-4047-b19e-c57a4785ab5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.096963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-69dfb8a8-ea50-4047-b19e-c57a4785ab5c None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:43.098346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-69dfb8a8-ea50-4047-b19e-c57a4785ab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:43.102631 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:43.161260 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b46c-60ff-4e58-9866-8a8a7f33eb55 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.178952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 434/878] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:43.187526 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b56d17-4b54-4eeb-99bd-c8d0bdf4519b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.188068 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b56d17-4b54-4eeb-99bd-c8d0bdf4519b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.188777 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b56d17-4b54-4eeb-99bd-c8d0bdf4519b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.191282 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 435/879] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:43.196586 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.196873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.197151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.206558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.207249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.207529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.207803 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.208557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:43.208864 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:43.210356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:43.219770 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 445/880] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:43.219770 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:43.221025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:43.222376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:43.260498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:43.309346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 436/881] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:43.309346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:43.348986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a74612-e7bb-4cdf-b025-ca6352a68d88 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.374592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:43.384481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.385165 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.385496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.385768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.386503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:43.387897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:43.403106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 446/882] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:35:43 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:43.404019 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:43.424332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e5ab3-35ef-4119-803b-0d463e1a16a7 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.470473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.471492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.471943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.472390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.473342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:43.473911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:43.475716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:43.483543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 437/883] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:43.555997 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:43.565287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c9869-9dbf-4928-8e9a-6ba5bd84c8d9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.607889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.608757 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.609064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.609371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.623065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:43.631740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:43.632582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:43.633547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 447/884] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:35:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:43.639457 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:43.759269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed19fb-f925-4971-8923-ec803ed5981c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.788974 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:43.795376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.796064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.796340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.796612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.797383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:43.797684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:43.799070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:43.808828 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 438/885] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:35:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:43.814327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:43.921501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33005dd5-aa8b-4236-850d-38b722729651 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:43.957932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:43.958776 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:43.959056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:43.959331 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:43.970608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:43.972106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:43.990953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:43.991774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:43.992561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:43.992561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:43.992561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:43.992561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:43.996970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 448/886] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:43.998347 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:44.105126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fd26c-ed08-4593-8975-6c78142f0a79 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:44.140021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:44.141095 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:44.141756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:44.142325 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:44.143444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:44.143944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:44.146270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:44.164098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 439/887] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:44.165968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:44.186838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:44.272236 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2638a3d-e57d-4e68-8c8f-477461a88335 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:44.309442 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449759-5066-4b99-9749-d100125d42ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:44.310278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449759-5066-4b99-9749-d100125d42ba None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:44.310565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449759-5066-4b99-9749-d100125d42ba None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:44.310839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449759-5066-4b99-9749-d100125d42ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:44.319240 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36449759-5066-4b99-9749-d100125d42ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:44.320751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36449759-5066-4b99-9749-d100125d42ba None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-36449759-5066-4b99-9749-d100125d42ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:44.336518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:44.337438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:44.338266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:44.341722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 449/888] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:35:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:44.344570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:44.400813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:44.448654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a24a5-c2c2-40b0-92c9-8603e0558cdd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:44.483337 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:44.484018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:44.484299 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:44.484573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:44.485442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:44.485751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:44.487195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a3716-b867-4c7f-ab39-5f4fcb3c6b13 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:44.515333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 440/889] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:35:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:44.522597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449759-5066-4b99-9749-d100125d42ba None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:44.597598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:44.633015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449759-5066-4b99-9749-d100125d42ba None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:44.673155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:44.673871 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:44.674167 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:44.674437 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:44.752984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:44.754438 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:44.822225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:44.832769 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f53532-2429-48cf-92e5-9e87d77bbda6 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:44.833069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f53532-2429-48cf-92e5-9e87d77bbda6 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:44.833478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f53532-2429-48cf-92e5-9e87d77bbda6 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:44.835308 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 441/890] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:44.840374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:44.840662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:44.840943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:44.858319 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:44.859541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:44.864506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 450/891] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:44 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:35:44.865886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:44.937343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 442/892] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:44.963580 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:44.975910 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b10c21-8c3a-442c-aa6c-08dfd78ab683 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.013734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.014616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.015002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.015374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.021591 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.028928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.030434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:45.056034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:45.056825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:45.057735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:45.057735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:45.057735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:45.057735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:45.061194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 451/893] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:45.070514 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.093932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.096386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb5310-bbeb-4256-87e2-03fd93fde65f None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.132293 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.133221 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.133678 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.134021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.135059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.136451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:45.150105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:45.151034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:45.151767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 443/894] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:45.157316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.186251 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505bdb-42bc-4d4c-a782-a271522d2891 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.223395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.224139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.224519 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.224823 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.225596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:45.225896 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.227495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.240558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 452/895] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:45.246258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.291854 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2ec6-7d0c-47d1-8169-a1e1cdf31e91 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.294149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.327524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.328225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.328500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.328768 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.329513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:45.329813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.331199 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.336617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 444/896] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:45.346039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.347242 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.375183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d32bc1-144c-4c04-ae69-16d50729d586 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.424623 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.425864 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.426353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.427083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.436699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.439072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:45.460395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:45.461267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:45.462073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:45.464416 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 453/897] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:45.470882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.473691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31999f0-49c1-4409-aa01-3e65b9caf093 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.511596 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.512386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.512672 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.512943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.514517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:45.514517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.515908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.526696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 445/898] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:45.527595 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.562288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.591247 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc204-b540-4a61-ae31-465dc72c2ae1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.628916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.630020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.630529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.630999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.632164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:45.632715 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.634464 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.643579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeefa2-e81d-4eb4-b7ee-d796e17b0ed2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.644572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.652781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 454/899] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:45.655447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.686696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.687416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.687699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.687974 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.688975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] RBAC: Adding query filter params (interface=public, service_id=ab381b03991b45f7a83117b3130618b7, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:45.689202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.690707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.701910 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 446/900] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/endpoints?service_id=ab381b03991b45f7a83117b3130618b7&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:45.703536 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:45.778705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac9338-0e86-48e1-8515-f2d25bb5a5cb None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.802696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:45.808495 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ad2e6-1128-46e4-b6e1-d4296ac3d73b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:45.818120 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.819125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.819620 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.820149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.821181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] RBAC: Adding query filter params (role.id=d2408e7e1e3241d6ae009ae98520c593, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=45013317d2524d2a87ff2cc558b409fe) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:45.821714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.823598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9de9b-bc85-4270-b3b3-af35e630bc68 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.837460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 455/901] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/role_assignments?user.id=45013317d2524d2a87ff2cc558b409fe&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=d2408e7e1e3241d6ae009ae98520c593 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:45.844819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd5eb5-4ca5-423d-bfe3-9d32cb63e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:45.845554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5eb5-4ca5-423d-bfe3-9d32cb63e303 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.845895 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5eb5-4ca5-423d-bfe3-9d32cb63e303 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.846177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5eb5-4ca5-423d-bfe3-9d32cb63e303 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.846876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd5eb5-4ca5-423d-bfe3-9d32cb63e303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:45.848298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd5eb5-4ca5-423d-bfe3-9d32cb63e303 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:45.855669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 447/902] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:45 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:35:45.877309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5fb5-22c7-4d65-8050-d02074e23f3f None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.877691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5fb5-22c7-4d65-8050-d02074e23f3f None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.877867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5fb5-22c7-4d65-8050-d02074e23f3f None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.878111 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4224085-792c-4353-87c8-4ae333e2ef7d None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.878714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4224085-792c-4353-87c8-4ae333e2ef7d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.879219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4224085-792c-4353-87c8-4ae333e2ef7d None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.879654 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 448/903] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:45.881738 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 456/904] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:45.886079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.886692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.887208 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.890183 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:45.890883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:45.891421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:45.907961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:45.909113 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:45.913443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:45.915062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:45.996074 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 449/905] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:46.005767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:46.013005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 457/906] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:46.025509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:46.038421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:46.143331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:46.156664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b192-cefc-4b19-891e-c0fb78671071 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:46.161029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ddb2d-6331-4cb9-8d61-eb08974b422b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:46.197899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:46.198589 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:46.198870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:46.199401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:46.200106 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:46.201641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:46.205792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:46.206926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:46.207401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:46.207861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:46.208782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:46.210493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:46.224356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:46.225301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:46.226134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:46.226871 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 450/907] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:35:46 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:46.230328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 458/908] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:35:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:46.237045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:46.376539 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840df5d0-69e3-4e1d-a659-770f221bdbf7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:46.417130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:46.418224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:46.418718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:46.419197 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:46.419916 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:46.420221 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:46.421640 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:46.427289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 451/909] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:46.433350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:46.509191 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:46.552019 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c05070-c7ad-431f-9448-3f276c8ad9db None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:46.587083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:46.587768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:46.588048 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:46.588320 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:46.589035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:46.589377 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:46.590783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:46.596307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 459/910] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:46.602351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:46.616676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:46.723267 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21827dd-b7b3-4f0b-add9-a763bb626e78 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:46.759585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:46.760269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:46.760545 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:46.760871 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:46.761699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] RBAC: Adding query filter params (interface=public, service_id=5b36bb08c27b4eef89ee02505d1469ee, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:46.762104 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:46.763494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1c346-1602-4ec4-80b3-ba7674f4337e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:46.769627 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 452/911] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:35:46 2022] GET /identity/v3/endpoints?service_id=5b36bb08c27b4eef89ee02505d1469ee&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:46.778360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:46.885392 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59180a71-5641-4480-aad5-c92689389b99 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:46.908476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:46.922101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865a99c-c55a-4f85-8672-dbb50e43fcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:46.922789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865a99c-c55a-4f85-8672-dbb50e43fcbb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:46.923472 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865a99c-c55a-4f85-8672-dbb50e43fcbb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:46.923747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865a99c-c55a-4f85-8672-dbb50e43fcbb None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:46.924441 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865a99c-c55a-4f85-8672-dbb50e43fcbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:46.925872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865a99c-c55a-4f85-8672-dbb50e43fcbb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:46.932443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 460/912] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:46 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:35:48.149083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af972607-27ff-463a-a5f8-6bb035d365ee None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.149643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af972607-27ff-463a-a5f8-6bb035d365ee None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.149718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af972607-27ff-463a-a5f8-6bb035d365ee None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.151773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 453/913] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:48.157539 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.157951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.158139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.176112 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:48.177476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:48.214542 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256654ea-ac73-4ffb-b45d-259b110d770b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.214937 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256654ea-ac73-4ffb-b45d-259b110d770b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.215120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256654ea-ac73-4ffb-b45d-259b110d770b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.216713 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 454/914] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:48.221886 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.222173 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.222449 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.239662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:48.240858 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:48.250359 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 461/915] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:48.275169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.319454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 455/916] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:48.341140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.344207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:48.398471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89dee5-31fb-4a68-94da-daa244a82d31 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:48.422955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:48.442976 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:48.443813 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.444095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.444719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.455278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:48.458604 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0627dc4-eb94-4007-b35e-6efc22671309 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:48.463538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:48.464510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:48.465352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 462/917] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:48.474121 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.497561 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:48.498376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.498653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.499231 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.499596 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:48.500976 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:48.513519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:48.514412 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:48.515150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 456/918] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:48.520879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.590826 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:48.603863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b2df-01e9-4288-9262-bb979094a3e3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:48.630077 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d01c6-98fa-49a9-9905-efe67b3fbc30 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:48.638830 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:48.639501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.639775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.640045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.640798 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:48.641089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:48.642507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a47e4-7a04-41dc-9bb0-270f3fccb455 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:48.651294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 463/919] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:48.652695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5a5ee-aaf8-481f-bba2-9aad5314172c None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.652979 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5a5ee-aaf8-481f-bba2-9aad5314172c None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.653265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5a5ee-aaf8-481f-bba2-9aad5314172c None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.654477 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 464/920] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:48.660195 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.660476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.660750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.672359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:48.673116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.673356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.673630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.674339 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:48.674644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:48.676029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:48.678454 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:48.679630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:48.681479 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 457/921] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:48.686433 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.712249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:48.756448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 465/922] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:48.757345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.814797 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be4472-e8cd-47b4-b38d-cd66f904ab11 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:48.849669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:48.857795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:48.859288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.859755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.860232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.871885 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032b2c5-9c10-47be-a3f4-44d406ce7ddd None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:48.876053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:48.877929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:48.895912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:48.896968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:48.899063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:48.899063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:48.899063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:48.899063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:48.899063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 458/923] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:48.899063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.912641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:48.913653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:48.913653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:48.913893 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:48.914608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:48.914913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:48.916307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:48.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 466/924] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:48.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:48.957552 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.014820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7ebf2-4e6a-458c-8797-52984d6c9e46 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.034452 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc40a8-f0f3-48ad-9276-91ab51e1b881 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.057977 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.058942 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.059280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.059717 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.060707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.062604 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.071375 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.072036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.072323 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.072591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.073344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:49.073646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.075038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:49.076911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:49.077821 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:49.078593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 459/925] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:49.079338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.096821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 467/926] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:49.096821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.138475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.191301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.195040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa93c0-f3a6-4743-b26d-407beeb04322 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.201407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530ae0-edde-47b6-b504-e32b775eec54 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.229684 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdce7a-ffea-47be-a685-1b9bd4c5b2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.230381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdce7a-ffea-47be-a685-1b9bd4c5b2f0 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.230662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdce7a-ffea-47be-a685-1b9bd4c5b2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.231060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdce7a-ffea-47be-a685-1b9bd4c5b2f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.231977 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdce7a-ffea-47be-a685-1b9bd4c5b2f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.233609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdce7a-ffea-47be-a685-1b9bd4c5b2f0 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.234300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.234964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.235240 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.235508 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.236206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:49.236504 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.237923 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.242608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 468/927] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:49.243378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.260018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 460/928] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:35:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:49.261510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82403229-44e6-41a9-8437-63deee2e00db None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.261801 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82403229-44e6-41a9-8437-63deee2e00db None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.262080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82403229-44e6-41a9-8437-63deee2e00db None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.263333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 461/929] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:49.264147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82403229-44e6-41a9-8437-63deee2e00db None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.367199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c3c60-3cf9-44cf-9fbd-7642607ca083 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.390498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.397568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82403229-44e6-41a9-8437-63deee2e00db None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.405043 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.405929 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.406214 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.406486 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.415285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.416765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-56e7e5cb-3962-4c3b-9d48-8e60a1c12e8e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:49.433109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:49.434093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:49.434984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 469/930] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:49.436228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.436228 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.436395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.448388 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.448997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.450369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.450369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.450369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:49.450584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.452523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.458630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:49.458630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:49.462486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 462/931] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:49.462486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.471733 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.534873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 470/932] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:49.535550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.579638 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5a961-e469-4936-950e-b7a56d875e27 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.617221 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.618569 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.618569 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.618569 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.619299 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:49.619606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.621052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.639422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 463/933] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:49.640332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.654245 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325cca7-6266-4d99-84b7-96bcf12ba5e6 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.666565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.690921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.691608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.691893 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.692171 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.693064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] RBAC: Adding query filter params (interface=public, service_id=8aa5f6b8527c4134bea9e2d16a9a626d, region_id=RegionOne) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:49.693424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.694862 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.703586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 471/934] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/endpoints?service_id=8aa5f6b8527c4134bea9e2d16a9a626d&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:49.704397 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.719073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.756435 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e829e4-5f9b-4ec1-87cd-7688ee9feea9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.795144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.796094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.796495 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.796766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.797452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.798833 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.812946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41024e9-df01-4b47-b60a-ffdafe45b636 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:49.813914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:49.814758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:49.815506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 464/935] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:49.816270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.854294 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.855951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.856234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.856511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.868887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.870485 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:49.888311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:49.890132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:49.891250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:49.891250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:49.891250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:49.891250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:49.891250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 472/936] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:49.891250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:49.921033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.937893 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c8156-d1fe-4373-95ad-dbc1017595fd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:49.966284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:49.973385 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:49.974164 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:49.974437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:49.974735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:49.975420 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:49.976799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:49.982926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 465/937] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:35:49.990300 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:50.022768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c6ca0-be52-4749-8d9e-445c874a12a4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.067833 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.068337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.068680 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.069015 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.069807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:50.070178 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.071692 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.077919 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 473/938] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:50.084153 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:50.130036 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c973f5-5d12-4dd4-97a5-f973f1ce154c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.172605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.173343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.173615 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.173889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.174629 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:50.174931 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.176372 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.196140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 466/939] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:50.197510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:50.214148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:50.224828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c0823-042e-4724-bdf9-f1da0cad42f7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.256691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:50.268498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.269796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.270210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.270502 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.271509 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:50.271829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.275282 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.284346 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 474/940] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:50.290842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:50.331914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38178bcb-16b6-414a-9842-80517d878a86 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.376358 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.377178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.377507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.377798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.387777 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.389230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.406032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b940d-5832-45fd-8348-587f20b679b3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:50.406553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:50.407353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:50.408177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:50.410812 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 467/941] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:50.421014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:50.448315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.449280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.449573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.449849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.450808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.452840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.479252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 475/942] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:35:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:50.505955 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:50.547119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f423438-c6b0-4bd1-9d0d-33d96faa0bcd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.595542 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.596229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.596506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.596786 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.597621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:50.597930 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.599549 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5a19f-64c7-4dcf-905c-ac5c44d2c9f4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.614116 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 468/943] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:50.620264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:50.676984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:50.739591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd823cf-1aea-408c-b5d6-f655cc368f02 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:50.774968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10591e37-c96f-45e3-be44-32a08d017d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:50.775885 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10591e37-c96f-45e3-be44-32a08d017d56 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:50.776174 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10591e37-c96f-45e3-be44-32a08d017d56 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:50.776447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10591e37-c96f-45e3-be44-32a08d017d56 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/45013317d2524d2a87ff2cc558b409fe/roles/d2408e7e1e3241d6ae009ae98520c593` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:50.778776 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10591e37-c96f-45e3-be44-32a08d017d56 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=45013317d2524d2a87ff2cc558b409fe, role_id=d2408e7e1e3241d6ae009ae98520c593)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:50.780235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10591e37-c96f-45e3-be44-32a08d017d56 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:50.794944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 476/944] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:35:50 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/45013317d2524d2a87ff2cc558b409fe/roles/d2408e7e1e3241d6ae009ae98520c593 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:35:51.548195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77803-8be0-4d54-9548-859ffc7b5730 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:51.548694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77803-8be0-4d54-9548-859ffc7b5730 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:51.548798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77803-8be0-4d54-9548-859ffc7b5730 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:51.550409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 469/945] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:51.556086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:51.556477 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:51.556642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:51.573713 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:51.574911 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:51.649353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 477/946] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:51.666176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77803-8be0-4d54-9548-859ffc7b5730 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:51.776767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77803-8be0-4d54-9548-859ffc7b5730 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:51.804080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:51.813394 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:51.814333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:51.814810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:51.815356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:51.816269 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:51.818170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:51.819864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: WARNING keystone.server.flask.common [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:35:51.838423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.sql.core [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:35:51.838423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:35:51.838423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [parameters: {'id': 'ed4d8d2581694b489978bb7d23206bfe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:35:51.838423 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150372) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed4d8d2581694b489978bb7d23206bfe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed4d8d2581694b489978bb7d23206bfe. May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application conn.query(q) May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:35:51.856273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application result.read() May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:35:51.857105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application next(self.gen) May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application yield resource May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application next(self.gen) May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise self.value May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application session.commit() May 19 14:35:51.858037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self.session.flush() May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application compat.raise_( May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:35:51.858864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:35:51.860427 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application conn.query(q) May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application result.read() May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:35:51.861176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [parameters: {'id': 'ed4d8d2581694b489978bb7d23206bfe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:51.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:51.862820 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed4d8d2581694b489978bb7d23206bfe. May 19 14:35:51.864544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:51.867120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 470/947] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 14:35:51 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:51.873330 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:51.908559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:51.977105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6846c-99f9-4c33-82df-ac9f6a7b6c14 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:52.018417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:52.019235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.019524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.019800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.028421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:52.030142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-ed5eb21b-f998-4f39-aa8b-d13711ccf249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:52.047971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:52.048873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:52.049743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:52.050233 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 478/948] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:52.058812 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:52.165019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a490d3-259c-4b8d-9641-b28de6dca9e3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:52.210058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:52.211122 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.211576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.212023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.213260 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:52.213786 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:52.215723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51413d15-3780-47c4-a23d-db8015b3d6da None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:52.243843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 471/949] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:52.388289 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485057a2-672e-4b21-ae04-b0a662107d53 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.388625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485057a2-672e-4b21-ae04-b0a662107d53 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.388869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485057a2-672e-4b21-ae04-b0a662107d53 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.390582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 479/950] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:52.396589 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.397162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.397658 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.417686 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:52.419357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:52.459409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:52.495544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 472/951] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:52.520745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485057a2-672e-4b21-ae04-b0a662107d53 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:52.625767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:52.634915 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485057a2-672e-4b21-ae04-b0a662107d53 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:52.672857 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:52.673833 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.674180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.674459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.675299 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:52.676928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:52.691688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:52.692788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:52.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 480/952] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:52.699336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:52.806083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21322-0520-440b-8dc4-62d0553a73de None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:52.845674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:52.846628 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.847051 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.847476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.848326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:52.848837 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:52.850500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65950da-aa67-409e-bc5b-be3901742e48 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:52.856455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 473/953] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:52.862830 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:52.964943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ee172-e4e0-4774-a46a-2fecb8cdba20 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.965616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ee172-e4e0-4774-a46a-2fecb8cdba20 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.966093 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ee172-e4e0-4774-a46a-2fecb8cdba20 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.968087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 474/954] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:52.973502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:52.974110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:52.974590 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:52.975029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d754ce-bb2b-4cbc-b27e-6f57a9cb010d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:52.997971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:53.000561 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:53.015592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.016374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.016742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.017088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.018132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:53.018250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.019830 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.026458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 481/955] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:53.032154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.042147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:53.072176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:53.101064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 475/956] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:53.120014 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.171055 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5849c1-8c83-469c-9d70-e617077f11bc None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.207012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7858b-f2a7-49b4-8609-f1884a4982e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.207683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7858b-f2a7-49b4-8609-f1884a4982e5 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.207963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7858b-f2a7-49b4-8609-f1884a4982e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.208293 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7858b-f2a7-49b4-8609-f1884a4982e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.209031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7858b-f2a7-49b4-8609-f1884a4982e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.210456 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7858b-f2a7-49b4-8609-f1884a4982e5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.230614 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 482/957] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:35:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:35:53.231863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70262a6-50c1-4992-b1fd-9021a8483506 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.232138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70262a6-50c1-4992-b1fd-9021a8483506 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.232411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70262a6-50c1-4992-b1fd-9021a8483506 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.233887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 483/958] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:53.240304 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.240594 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.240868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.252174 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428d1a2-7300-4caf-acca-26d40594b3aa None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.263827 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:53.265145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:53.312608 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.313464 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.313738 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.314009 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] PATH_INFO: `/v3/services/compute` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.314662 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.316059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:53.333230 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:53.334306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:53.335870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 476/959] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:53.342404 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.373025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 484/960] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:53.384343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:53.396483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.410425 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:53.486643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb55fb-493c-4a7a-91ab-0f22a1326001 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.517690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c26e8-2c0b-45ea-ab7d-906ba998d47a None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.531560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.532243 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.532643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.533013 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.533977 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] RBAC: Adding query filter params (name=compute) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:53.534281 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.535827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.542128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 477/961] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:53.548017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.565849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.567164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.567644 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.568304 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.582863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:35:53.591243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:53.592296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:35:53.593267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:53.595328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 485/962] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:53.602572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.664381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:53.674553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc66a4-e15e-4294-9c7c-36a4aa9abeff None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:53.710431 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.711351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.711351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.711640 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.712343 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] RBAC: Adding query filter params (type=compute) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:53.712641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.714078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.725634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 478/963] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:53.736634 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff4e20-9c31-486a-8d62-7848a5e98895 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.776443 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.777565 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.778069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.778551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.779628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:53.780140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.781895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.792628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 486/964] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:53.798728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:53.913098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f925d-c6a4-438b-823f-79eb0744da16 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:53.951445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:53.952258 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:53.952528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:53.952752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:53.963647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:53.965093 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:53.983311 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:53.984187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:53.985017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:53.985017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:53.985017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:53.985017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:53.985017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 479/965] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:53.991597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:54.097564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57718f86-cdd4-4054-9f82-41c93043b14d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:54.126797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:54.132091 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:54.132783 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.133050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.133353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.134082 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:54.134393 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:54.135791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:54.152685 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 487/966] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:54.158454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:54.167982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:54.265671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4298ab6-1e04-4238-afdc-47ef3e82fa33 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:54.300146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:54.300987 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.301286 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.301562 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.309587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:54.311049 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:54.326000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:54.326902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:54.327760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 480/967] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:35:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:54.335215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:54.441858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aa81a-09a1-4bea-8c89-8adb462109ae None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:54.481735 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:54.482781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.483266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.483843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.484903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:54.485436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:54.487189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7d707-71e4-4dcc-bb09-63c64f542c56 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:54.508400 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 488/968] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:35:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:54.520760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:54.590111 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:54.626556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76616e-507c-4480-bc9c-305c05c5add1 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:54.663124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:54.663850 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.664142 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.664441 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.694584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d3f40-e190-43f5-93e0-e8d69b52f81a None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.694873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d3f40-e190-43f5-93e0-e8d69b52f81a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.695257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d3f40-e190-43f5-93e0-e8d69b52f81a None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.697686 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 489/969] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:54.703031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.703582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.703582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.721274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:54.722567 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:54.747624 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:54.749930 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:54.781954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:54.802496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 490/970] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:54.826373 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:54.829155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:54.875419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 481/971] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:35:54 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:35:54.881794 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:54.930719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd4583-ef81-4cfc-baac-9b3a1272ee67 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:54.982046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:54.982913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:54.983659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:54.983977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:54.985036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:54.987308 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:54.988741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: WARNING keystone.server.flask.common [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:35:55.002590 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8619f0e-0a3c-449a-be4d-e0771567c87d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.007458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.sql.core [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:35:55.007458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:35:55.007458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [parameters: {'user_id': 'ad68b7b94e204f248c4ac4a9fe51656c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:35:55.007458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application conn.query(q) May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result.read() May 19 14:35:55.018461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:35:55.019329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application next(self.gen) May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application yield resource May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application next(self.gen) May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise self.value May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application session.commit() May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:35:55.020272 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.session.flush() May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application compat.raise_( May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:35:55.021540 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:35:55.023235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application conn.query(q) May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application result.read() May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:35:55.024662 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ad68b7b94e204f248c4ac4a9fe51656c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:55.027764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:35:55.028937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 491/972] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:35:54 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:55.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.050596 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.051764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.052203 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.052575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.066532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.070645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:35:55.093923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:55.095630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:55.097192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:35:55.097192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:35:55.097192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:35:55.097192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:55.098128 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:55.101522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 482/973] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:55.102525 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.128462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:55.183807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d68d1-6166-4106-9eba-b9df895cc68a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.231583 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56234d-0aa3-47f3-9584-21572163751d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.231782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.232876 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.233615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.234085 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.245375 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.247434 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:35:55.268016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:55.269188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:35:55.270403 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:55.274123 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 492/974] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:55.281886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.288852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.289879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.290362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.290849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.291816 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:55.292342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.294168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.307417 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 483/975] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:55.313150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.396586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cc6c-5334-4292-b4e7-b1536c38638e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.399114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:55.401861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:55.431752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff686d9-83c6-4f80-89c5-286b3935147c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.438754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.439938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.440473 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.441030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.442206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:55.442794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.444739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.478784 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.479651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.479928 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.480204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.488760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.490351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.493090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 493/976] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-049b0296-b5c2-4a9b-be19-5af8a0307fd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:35:55.518188 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:55.519069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:35:55.519901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:55.522773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 484/977] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:55.529191 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.634357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381ec8-2cc1-42b5-8429-1e6a27fd1849 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.634827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381ec8-2cc1-42b5-8429-1e6a27fd1849 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.635224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381ec8-2cc1-42b5-8429-1e6a27fd1849 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.637127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 485/978] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:55.643147 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca09349-f8f7-41d1-af42-6cff676db289 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.643440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.643726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.644005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.663708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:55.665024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:55.689063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.690181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.690666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.691624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.692697 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:55.693458 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.695485 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.712641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 494/979] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:55.721327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.749028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:55.756064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 486/980] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:55.778223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:55.851081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de31a3-67b5-44dc-a9d9-47e37b627222 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.859994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:55.897603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.898273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.898569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.899161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.900006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] RBAC: Adding query filter params (role.id=d2408e7e1e3241d6ae009ae98520c593, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=45013317d2524d2a87ff2cc558b409fe) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:55.900324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.901821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.903506 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326194-3e90-43ff-af78-586d99d54015 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:55.909173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 495/981] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/role_assignments?user.id=45013317d2524d2a87ff2cc558b409fe&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=d2408e7e1e3241d6ae009ae98520c593 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:55.944945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:55.946072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:55.946525 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:55.946950 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:55.947773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:55.949435 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:55.962982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:55.964165 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:55.965211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:55.967522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 487/982] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:55.974055 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:56.072632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:56.091728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcd62e-c9df-40ef-a50d-726badd4867e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:56.133164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:56.134024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.134394 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.134762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.135569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:56.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:56.137504 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e04e19-ae20-461d-b28e-97aafb7b7987 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:56.143491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 496/983] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:56.149912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:56.194845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ac6a-fede-4b54-a9b0-590cb98cd66f None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.195207 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ac6a-fede-4b54-a9b0-590cb98cd66f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.195424 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ac6a-fede-4b54-a9b0-590cb98cd66f None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.196803 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 497/984] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:56.202052 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.202326 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.202602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.221689 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:56.223239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:56.260669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339091a5-4532-46be-aed7-2f64bcd4b7f8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:56.294544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:56.301285 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 498/985] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:56.305127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:56.305840 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.306115 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.306383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.307081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:56.307383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:56.308762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:56.318815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 488/986] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:56.319050 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:56.319719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:56.398335 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:56.422560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91521e5b-e2c5-4b81-8e07-bcd8ef4b06b9 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:56.425517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7c884-d96b-46e8-9cb4-b539ede74a21 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:56.458268 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a6d9e-6a33-4a61-90b8-8aef84e53633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:56.458943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a6d9e-6a33-4a61-90b8-8aef84e53633 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.459281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a6d9e-6a33-4a61-90b8-8aef84e53633 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.459558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a6d9e-6a33-4a61-90b8-8aef84e53633 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.460303 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a6d9e-6a33-4a61-90b8-8aef84e53633 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:56.461750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a6d9e-6a33-4a61-90b8-8aef84e53633 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:56.461847 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:56.462494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.462793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.463058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.463844 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] RBAC: Adding query filter params (interface=admin, service_id=8aa5f6b8527c4134bea9e2d16a9a626d, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:56.464155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:56.465566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:56.476365 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 489/987] 10.222.0.73 () {56 vars in 1249 bytes} [Thu May 19 14:35:56 2022] GET /identity/v3/endpoints?service_id=8aa5f6b8527c4134bea9e2d16a9a626d&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:56.476535 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 499/988] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:35:56 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:35:56.478106 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:56.554179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:56.599584 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483ab7b-ee6d-447a-ac84-15b5f50c8a16 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:56.648328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7642-75b6-4251-8871-1d59d8c7f5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:56.649473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7642-75b6-4251-8871-1d59d8c7f5fd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:56.649697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7642-75b6-4251-8871-1d59d8c7f5fd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:56.650037 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7642-75b6-4251-8871-1d59d8c7f5fd None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:56.650719 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c7642-75b6-4251-8871-1d59d8c7f5fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:56.652185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c7642-75b6-4251-8871-1d59d8c7f5fd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:56.669268 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 490/989] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:56 2022] GET /identity/v3/services => generated 1727 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:35:58.214317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab36ef9-512b-4777-9652-50e2959c44d0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.214854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab36ef9-512b-4777-9652-50e2959c44d0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.214854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab36ef9-512b-4777-9652-50e2959c44d0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.216407 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 500/990] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:35:58.222096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.222541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.222720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.241856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:58.244059 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:58.316627 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 491/991] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:35:58.338591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab36ef9-512b-4777-9652-50e2959c44d0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:58.448065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab36ef9-512b-4777-9652-50e2959c44d0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:58.485935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:58.486708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.486989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.487264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] PATH_INFO: `/v3/services/image` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.487973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:58.489522 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:58.505275 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:58.506226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:58.507000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 501/992] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:35:58 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:58.509500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:58.571187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:58.627043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0457d-439f-4c6b-ac65-98175a2c97c2 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:58.666997 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:58.667907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.668261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.668610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.669352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] RBAC: Adding query filter params (name=image) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:58.669754 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:58.671852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40963536-a9eb-43ac-ae13-a17652421092 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:58.682901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 492/993] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:58.684376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:58.688013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:58.734783 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e89bcc-7c7a-4e5a-83eb-07de8bd7708c None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.735072 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e89bcc-7c7a-4e5a-83eb-07de8bd7708c None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.735517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e89bcc-7c7a-4e5a-83eb-07de8bd7708c None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.736961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 493/994] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:58.742646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.742900 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.743180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.760747 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:58.761970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:58.805141 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b92a8-773c-4283-ad1f-9107b7a44afa None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:58.840167 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:58.840883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.841169 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.841471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.845434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 494/995] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:58.845573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] RBAC: Adding query filter params (type=image) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:58.845573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:58.845573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48ca17-93ce-48a7-a4a2-5415e7150791 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:58.849606 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 502/996] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:35:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:35:58.865288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:58.900337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5592b66-287f-47ec-a7e1-e23779b671de None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.900652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5592b66-287f-47ec-a7e1-e23779b671de None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.900885 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5592b66-287f-47ec-a7e1-e23779b671de None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.902493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 503/997] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:35:58.907926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:58.908209 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:58.908484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:58.919422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:58.926960 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:35:58.928239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:35:58.980674 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31025-6ddc-4cd9-9665-32f779ca6be0 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:58.997984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:59.013290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 504/998] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:35:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:35:59.021083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.021931 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.022207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.022525 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.023187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.024955 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.033083 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:59.037932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:59.038856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:35:59.039619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 495/999] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:35:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:35:59.047361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.155875 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e515a-db22-454e-aa93-d85dc51b3f66 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.159082 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6d43a-83b4-443a-adf9-5ec1ae4ada1a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.168228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:59.197788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.198274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.198463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.198746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.199018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.199114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.199361 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.199629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.199782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] RBAC: Adding query filter params (name=compute) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:59.200061 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.200311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.201464 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.201712 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.206807 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 496/1000] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:59.212492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:35:59.214767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:35:59.216092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:35:59.216943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:35:59.218344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 505/1001] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:35:59.223018 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.288297 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:59.327127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d67c2b-d910-4730-8890-1e70dfc1f51e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.330280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757db4f0-7ebc-4584-a615-2195a5d45239 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.363897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.364685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.364925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.365189 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.365957 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] RBAC: Adding query filter params (type=compute) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:59.366239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.366361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.366923 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.367205 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.367481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.367641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.368191 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:59.368498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.369985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.373762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 497/1002] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:59.383402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 506/1003] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:59.383543 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.384241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.419286 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:59.490043 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c8d0-b86f-4f34-b57a-51a56367200b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.491909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900bb5-6117-4bc1-839b-adc073fe3e40 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.522470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:59.534407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.535103 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.535384 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.535646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.536503 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] RBAC: Adding query filter params (interface=public, service_id=d3005d3335db4fd5a6c814af81057876, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:59.536819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.536908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.537554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.537842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.538115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.538263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.538868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:35:59.539125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.540501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.544034 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 498/1004] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/endpoints?service_id=d3005d3335db4fd5a6c814af81057876&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:59.549127 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.554761 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 507/1005] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:35:59.555524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:35:59.662918 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:35:59.663466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6f09f-7a24-4e14-823f-52b9e527c4ab None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.667788 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39615f-f42c-4ef1-9a2b-2db4afdc1401 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:35:59.699808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0ef69-5387-415e-b3ae-b728de169e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.700483 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0ef69-5387-415e-b3ae-b728de169e1b None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.700765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0ef69-5387-415e-b3ae-b728de169e1b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.701042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0ef69-5387-415e-b3ae-b728de169e1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.701829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0ef69-5387-415e-b3ae-b728de169e1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.703232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0ef69-5387-415e-b3ae-b728de169e1b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.707236 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2d79-12a3-4f5d-8818-9db0f538bf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:35:59.708202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d79-12a3-4f5d-8818-9db0f538bf4b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:35:59.708679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d79-12a3-4f5d-8818-9db0f538bf4b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:35:59.708995 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2d79-12a3-4f5d-8818-9db0f538bf4b None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:35:59.709817 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2d79-12a3-4f5d-8818-9db0f538bf4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:35:59.711524 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2d79-12a3-4f5d-8818-9db0f538bf4b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:35:59.719166 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 499/1006] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:35:59 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:35:59.720073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 508/1007] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:35:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:36:01.089994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481dda0-0286-4b58-b6b2-23df26a8542f None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:01.090516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481dda0-0286-4b58-b6b2-23df26a8542f None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:01.090793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481dda0-0286-4b58-b6b2-23df26a8542f None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:01.092050 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 500/1008] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:01.097461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7551e-f360-423e-b89a-3837c8bf7a6b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:01.097868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7551e-f360-423e-b89a-3837c8bf7a6b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:01.097977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7551e-f360-423e-b89a-3837c8bf7a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:01.116062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-39a7551e-f360-423e-b89a-3837c8bf7a6b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:01.117704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-39a7551e-f360-423e-b89a-3837c8bf7a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:01.166022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:01.197114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 509/1009] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:01.219182 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481dda0-0286-4b58-b6b2-23df26a8542f None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:01.334522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481dda0-0286-4b58-b6b2-23df26a8542f None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:01.370282 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7321a0-f002-4d10-9255-099be65296f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:01.370964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7321a0-f002-4d10-9255-099be65296f4 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:01.371246 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7321a0-f002-4d10-9255-099be65296f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:01.371520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7321a0-f002-4d10-9255-099be65296f4 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:01.372324 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7321a0-f002-4d10-9255-099be65296f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:01.373785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7321a0-f002-4d10-9255-099be65296f4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:01.397307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 501/1010] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:36:01 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:01.981529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f40370-d5d7-411f-bf9e-a71df2777af3 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:01.981812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f40370-d5d7-411f-bf9e-a71df2777af3 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:01.982086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f40370-d5d7-411f-bf9e-a71df2777af3 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:01.983505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 510/1011] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:01.989040 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:01.989519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:01.990220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:01.991691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc2cb-0918-493b-94bf-d42b583330a0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:01.992007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc2cb-0918-493b-94bf-d42b583330a0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:01.992252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc2cb-0918-493b-94bf-d42b583330a0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:01.993628 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 511/1012] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:02.000938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.001309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.001596 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.008738 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:02.009958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:02.019531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:02.020938 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:02.029365 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:02.091763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 502/1013] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:02.101371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 512/1014] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:36:02.114679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:02.116908 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:02.233397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294f62b-4a22-4265-bbba-11d3d83454c7 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:02.234826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac0491-ce14-483c-a191-6b2887991f25 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:02.238138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:02.270943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:02.271789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.272069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.272341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] PATH_INFO: `/v3/services/compute` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.273000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:02.274878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:02.275131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:02.275944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.276225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.276494 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.277842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:02.279156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:02.285806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 513/1015] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:02.286828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:02.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:02.288375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:02.289475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 503/1016] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:36:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:02.294664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:02.312482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:02.391213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932e2b8-767e-49b0-b2f9-b34e075e4d8a None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:02.401623 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413ae52-dd3c-4b58-aad7-064e4d6b9b1c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:02.430211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:02.430872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.431146 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.431418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.432142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:02.433633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:02.436504 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:02.437170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.437480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.437759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.438456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:02.438762 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:02.440121 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:02.450032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 504/1017] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:36:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:02.451218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:02.465194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 514/1018] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:36:02 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:02.561780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:02.586883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9cecc-da96-4f22-8328-415b5acbb6a1 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:02.625104 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:02.625857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.626130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.626406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.627125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] RBAC: Adding query filter params (type=compute) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:02.627432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:02.628827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1f0a3-ee33-4e02-85b7-d5ba05470567 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:02.639500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 505/1019] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:36:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:02.640714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:02.670030 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:02.747571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7751f-8004-4607-a50e-aa02adfb0397 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:02.783401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6234-6402-4381-975a-7a154013c431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:02.783986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6234-6402-4381-975a-7a154013c431 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:02.784265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6234-6402-4381-975a-7a154013c431 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:02.784534 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6234-6402-4381-975a-7a154013c431 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:02.785324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6234-6402-4381-975a-7a154013c431 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:02.786768 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6234-6402-4381-975a-7a154013c431 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:02.806282 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 515/1020] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:36:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:36:03.684331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3cfd-da5c-4484-a81e-3d384abb687e None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:03.684879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3cfd-da5c-4484-a81e-3d384abb687e None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:03.684879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3cfd-da5c-4484-a81e-3d384abb687e None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:03.686481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 506/1021] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:03.692043 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:03.692448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:03.692633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:03.710098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:03.711341 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:03.789351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 516/1022] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:03.812731 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad3cfd-da5c-4484-a81e-3d384abb687e None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:03.919659 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad3cfd-da5c-4484-a81e-3d384abb687e None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:03.953836 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:03.954647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:03.954926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:03.955195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] PATH_INFO: `/v3/services/image` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:03.955850 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:03.957257 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:03.969424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:03.970307 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:36:03.971094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:03.973718 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 507/1023] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:36:03 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:03.977073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:03.985722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:04.087618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57294-75eb-415c-8b11-f2c8c2cf6445 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:04.123386 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:04.124059 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:04.124338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:04.124609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:04.125344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] RBAC: Adding query filter params (name=image) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:04.125645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:04.127044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:04.132398 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 517/1024] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:04.138131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:04.148934 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:04.244695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae1c75-af4e-4033-9393-3f2598c63177 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:04.278815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:04.279490 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:04.279764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:04.280030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:04.280776 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] RBAC: Adding query filter params (type=image) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:04.281042 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:04.282470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:04.287606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 508/1025] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:04.293420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:04.362829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:04.395551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc27b-2f74-4785-b8e2-7c2bedae9f5d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:04.430156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:04.430819 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:04.431098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:04.431367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:04.432156 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] RBAC: Adding query filter params (interface=public, service_id=7cfce4311f4e4ce6b873447cdc7111f2, region_id=RegionOne) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:04.432466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:04.433886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a6457-baed-4b4a-bd1d-91075b2069f3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:04.439309 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 518/1026] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:36:04 2022] GET /identity/v3/endpoints?service_id=7cfce4311f4e4ce6b873447cdc7111f2&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:04.444811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:04.524614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:04.557354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9f897-5c4d-42af-9319-c0ecef36be9a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:04.591602 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e77ac-8b4a-438d-a773-bcda1205504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:04.592293 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e77ac-8b4a-438d-a773-bcda1205504e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:04.592575 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e77ac-8b4a-438d-a773-bcda1205504e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:04.592844 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e77ac-8b4a-438d-a773-bcda1205504e None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:04.593565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e77ac-8b4a-438d-a773-bcda1205504e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:04.594973 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e77ac-8b4a-438d-a773-bcda1205504e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:04.601913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 509/1027] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:36:04 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:36:04.826918 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17edbf-7d3a-49ca-a2ff-746cef7f17da None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:04.827522 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17edbf-7d3a-49ca-a2ff-746cef7f17da None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:04.827522 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17edbf-7d3a-49ca-a2ff-746cef7f17da None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:04.828969 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 519/1028] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:04.834827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:04.835474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:04.835474 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:04.853475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:04.854700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:04.926606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 510/1029] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:04.953027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17edbf-7d3a-49ca-a2ff-746cef7f17da None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:05.067510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17edbf-7d3a-49ca-a2ff-746cef7f17da None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:05.102391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965817f-720d-489d-9501-9eff4e6c8079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:05.103345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965817f-720d-489d-9501-9eff4e6c8079 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.103652 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965817f-720d-489d-9501-9eff4e6c8079 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.103932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965817f-720d-489d-9501-9eff4e6c8079 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.105146 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965817f-720d-489d-9501-9eff4e6c8079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:05.106672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965817f-720d-489d-9501-9eff4e6c8079 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:05.108488 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 520/1030] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:05.114250 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:05.142704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88079c-a5b0-4911-9f22-e239676a60c6 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.142983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88079c-a5b0-4911-9f22-e239676a60c6 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.143257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88079c-a5b0-4911-9f22-e239676a60c6 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.144632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 521/1031] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:05.150207 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.150481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.150750 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.157840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:05.174840 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:05.176259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:05.201975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:05.246081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e049d4f-341c-4883-9a8e-de52cc746855 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:05.249756 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 522/1032] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:05.272975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:05.285555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:05.286569 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.287140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.287611 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.288617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:05.290392 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:05.313889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 511/1033] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:36:05 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:05.384095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d28ced-6ffb-42a8-8aeb-3aeab942607b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:05.420172 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:05.427624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:05.428476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.428759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.429102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.441295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:05.449888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:05.450855 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:05.451651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 523/1034] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:36:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:05.461294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:05.578102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d4421-9d6a-4492-981e-9926a9560596 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:05.601773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:05.616779 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:05.617554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.617836 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.618109 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.618859 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:05.619164 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:05.620532 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:05.633326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 512/1035] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:36:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:05.635258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:05.740265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f61f6-e319-466f-8b84-631f3add8c38 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:05.779124 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:05.779936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.780213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.780484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.791581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:05.793095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:05.810963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:05.811826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:05.812673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:05.812673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:05.812673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:05.812673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:05.812673 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 524/1036] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:05.818894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:05.925499 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a32d9-16df-4870-b273-dc10bfbc1044 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:05.961712 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:05.962378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:05.962700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:05.962996 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:05.963730 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:05.964023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:05.965440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:05.982271 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 513/1037] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:05.983996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:06.033762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:06.090992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974da81-7c3e-406e-9bcd-6e79ea95ab48 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:06.124408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:06.125212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:06.125537 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:06.125809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:06.133451 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:06.134928 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:36:06.149636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:06.150605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:36:06.151499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:06.155736 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 525/1038] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:06.157195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:06.204879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:06.263138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da26500-c117-482a-8faf-23781e2434ac None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:06.297685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:06.298360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:06.298637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:06.298908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:06.299740 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:06.300047 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:06.301546 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:06.319142 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 514/1039] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:36:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:06.324897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:06.429882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403eeee-9d6e-4c00-acc1-b4bfcb0b2b43 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:06.432244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:06.467600 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:06.468249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:06.468529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:06.468806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:06.549817 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:06.551302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:06.658416 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 526/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:06.660247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:06.766245 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:06.769945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733839ed-528c-4e12-87a3-baacbaf869ac None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:06.805512 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:06.806318 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:06.806594 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:06.806865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:06.817350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:06.818825 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-4c44b570-92e9-409a-8d0e-857d5b346edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:06.836520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:06.837378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:06.838227 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:06.838227 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:06.838227 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:06.838227 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:06.842510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 515/1041] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:06.844013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:06.848349 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:06.853211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584b52b-c151-46fd-bcb8-98549d3d34f8 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:06.853743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584b52b-c151-46fd-bcb8-98549d3d34f8 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:06.854189 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584b52b-c151-46fd-bcb8-98549d3d34f8 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:06.856272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 516/1042] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:06.861800 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:06.862080 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:06.862350 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:06.880592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:06.881856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:06.952484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 517/1043] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:06.958436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c48f-39a8-4155-a812-0b06c9978a69 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:06.975218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:06.999348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.000305 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.000580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.000846 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.002052 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:07.002379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.004207 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.017056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 527/1044] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:07.022803 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.063587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:07.085576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2aa0f-20a8-4398-ba0b-f921996e33cc None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.089484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:07.122556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.123359 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.123637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.123904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] PATH_INFO: `/v3/services/image` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.124553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.125986 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.130008 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d08469-7828-4a2c-8a8a-606e16495df5 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:07.137929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:07.138912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:07.139849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 518/1045] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:36:06 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:07.145535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.167592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.168401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.168678 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.168944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.178176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.180304 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:36:07.197881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:07.198828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:07.199748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 528/1046] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:07.206158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.258911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf13f3-279b-4a44-9c24-464eb8b62716 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.294460 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.295047 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.295331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.295605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.296336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] RBAC: Adding query filter params (name=image) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:07.296652 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.298038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.303013 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 519/1047] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:07.308772 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.320854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db37fd4-9ae4-41f1-b639-28152e40e366 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.322505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:07.346379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:07.357788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.358476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.358757 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.359108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.359932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:07.360241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.361788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.375732 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 529/1048] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:07.382027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.418252 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97477c81-f0fe-4cb7-9e10-a15fc764cdec None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.452632 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.453335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.453621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.453902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.454604 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] RBAC: Adding query filter params (type=image) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:07.454907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.456386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.466254 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 520/1049] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:07.467374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.494287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ed70-28f5-4369-b2f9-88ec0260b688 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.540873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.541842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.542125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.542396 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.543696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] RBAC: Adding query filter params (role.id=d2408e7e1e3241d6ae009ae98520c593, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=f5c6cd73a1234dde953e73fedc6440ee) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:07.544010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.545809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8fd19-3cf7-4734-8306-9c1b3fc1d2cb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.561286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 530/1050] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:07 2022] GET /identity/v3/role_assignments?user.id=f5c6cd73a1234dde953e73fedc6440ee&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=d2408e7e1e3241d6ae009ae98520c593 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:07.561494 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:07.587966 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22a18e-735e-48ec-aee7-3bce74ef00de None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.628180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463c31b-0678-4bd9-be03-5a768bd87bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.628875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463c31b-0678-4bd9-be03-5a768bd87bfa None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.629155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463c31b-0678-4bd9-be03-5a768bd87bfa None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.629459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463c31b-0678-4bd9-be03-5a768bd87bfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.630279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463c31b-0678-4bd9-be03-5a768bd87bfa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:07.631691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463c31b-0678-4bd9-be03-5a768bd87bfa None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:07.661297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 521/1051] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:36:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:07.681665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297e28-732e-4881-87ee-6cee9333d861 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.682013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297e28-732e-4881-87ee-6cee9333d861 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.682302 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297e28-732e-4881-87ee-6cee9333d861 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.684259 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 531/1052] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:07.690541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.690829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.691112 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.710306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:07.711644 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:07.802354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 522/1053] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:07.826564 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88297e28-732e-4881-87ee-6cee9333d861 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:07.882741 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:07.941194 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88297e28-732e-4881-87ee-6cee9333d861 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:07.980948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:07.981795 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:07.982073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:07.982341 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:07.993329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:08.000882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:08.002244 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:08.003056 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 532/1054] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:08.008461 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:08.066731 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:08.114791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bbbe2-a2d4-4f2a-93c8-27f1ed404552 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:08.151185 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:08.151873 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:08.152153 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:08.152424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:08.153179 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:08.153552 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:08.155110 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:08.169361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 523/1055] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:08.170522 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:08.275883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a97ccf-d598-4d5f-9c57-2bca90843491 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:08.285815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:08.316550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:08.317535 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:08.317760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:08.318029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] PATH_INFO: `/v3/users/2e8810a880414be7b8c0b6b1ec7202b4` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:08.332306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] RBAC: Authorizing `identity:get_user(user_id=2e8810a880414be7b8c0b6b1ec7202b4)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:08.333965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:08.340448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 533/1056] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:08 2022] GET /identity/v3/users/2e8810a880414be7b8c0b6b1ec7202b4 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:08.341699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:08.451641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aab3a3-25be-40f4-836a-f72b6ffa34e7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:08.469917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:08.489036 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:08.490006 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:08.490491 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:08.490967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:08.567526 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:08.569480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:08.677619 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 524/1057] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:08.680430 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:08.784843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb52238-a125-4207-8046-af78493811b3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:08.820667 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb5e97-eaf2-4078-9442-a339ef76964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:08.821935 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb5e97-eaf2-4078-9442-a339ef76964b None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:08.822459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb5e97-eaf2-4078-9442-a339ef76964b None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:08.822930 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb5e97-eaf2-4078-9442-a339ef76964b None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:08.824675 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb5e97-eaf2-4078-9442-a339ef76964b None admin] RBAC: Authorizing `identity:get_project(project_id=01c4756c2b02470786b8d715f60de387)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:08.827463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb5e97-eaf2-4078-9442-a339ef76964b None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:08.833651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 534/1058] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:08 2022] GET /identity/v3/projects/01c4756c2b02470786b8d715f60de387 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:08.836489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:08.857455 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:08.949732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c919e1-da0e-46c6-a0f3-ec34a6cf8b17 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:08.985005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:08.985714 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:08.986025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:08.986288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:08.987170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=01c4756c2b02470786b8d715f60de387, user.id=2e8810a880414be7b8c0b6b1ec7202b4) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:08.987492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:08.989055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882ad8d-ee14-4cc2-a62d-3cc79a6b6924 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:08.995103 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 525/1059] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:08 2022] GET /identity/v3/role_assignments?user.id=2e8810a880414be7b8c0b6b1ec7202b4&scope.project.id=01c4756c2b02470786b8d715f60de387&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:09.803346 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2dfc5-7b34-4e9c-b381-8615d9ff6a2c None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:09.803880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2dfc5-7b34-4e9c-b381-8615d9ff6a2c None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:09.803880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2dfc5-7b34-4e9c-b381-8615d9ff6a2c None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:09.805503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 535/1060] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:36:09.811587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97f4d-6826-4e44-9095-3e1e87362014 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:09.812170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97f4d-6826-4e44-9095-3e1e87362014 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:09.812170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97f4d-6826-4e44-9095-3e1e87362014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:09.830796 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-52d97f4d-6826-4e44-9095-3e1e87362014 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:09.832027 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-52d97f4d-6826-4e44-9095-3e1e87362014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:09.904171 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 526/1061] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:09.926708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2dfc5-7b34-4e9c-b381-8615d9ff6a2c None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:09.964031 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:09.999228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a95505-b5f7-441d-bffa-b14a0ac25e3f None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:09.999517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a95505-b5f7-441d-bffa-b14a0ac25e3f None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:09.999775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a95505-b5f7-441d-bffa-b14a0ac25e3f None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.005084 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 527/1062] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:10.006338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.006612 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.006884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.025617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:10.026802 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:10.034763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.034904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2dfc5-7b34-4e9c-b381-8615d9ff6a2c None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.071411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.072153 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.072435 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.072698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.083420 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:10.091961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:10.092958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:10.094237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 536/1063] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:36:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:10.099916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.107661 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 528/1064] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:10.131591 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.199850 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.207419 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afd8ff-2cab-42f8-aa9d-efa2ead30bd2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.238914 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe364c-0d82-45d5-a273-0f96ad882022 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.244261 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.244939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.245218 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.245546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.246399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:10.246708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.248102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.256881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 537/1065] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:10.262751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.277617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.278280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.278560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.278829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.290838 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.351057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.352544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.367319 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbcd1f-1438-4490-bf50-25a686d2d209 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.403493 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.404646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.404925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.405190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.416886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.418672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:10.436800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:10.437731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:10.438587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:10.438587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:10.438587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:10.438587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:10.438587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 538/1066] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:10.444168 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.455384 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.461275 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 529/1067] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:36:10.463340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.509838 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.564226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d95ce4-7fb1-4904-b569-3b42fbb05edb None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.574637 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5746f-98cf-4d58-8cc0-9544a608bccd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.603276 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.603885 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.604166 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.604436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.605159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:10.605491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.607012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.612045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.613125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.613489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.613994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.627903 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 539/1068] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:10.629058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.632371 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.633879 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:10.654676 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:10.655568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:10.656585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:10.656585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:10.656585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:10.656585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:10.656585 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 530/1069] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:10.661962 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.705369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.740581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fddf8-3592-4d64-808d-7737a59055db None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.761638 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.769218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452440bc-cb69-4d7c-ac90-b904785c4fe6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.777714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.778748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.779032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.779303 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.787010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.788485 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:36:10.803468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:10.804790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:10.805720 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 540/1070] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:10.807755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.808475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.808798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.809092 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.809933 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:10.810243 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.810568 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:10.811727 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.834056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 531/1071] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:10.927072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8edaa2-8a91-45e0-89c4-0e86db226052 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:10.958685 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:10.968440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:10.969143 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:10.969470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:10.969760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:10.970617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:10.970924 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:10.972431 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:10.992867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 541/1072] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:10.998770 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.112422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e506455-b9e4-40b2-9f31-24756a4eae93 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.147008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.147815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.148091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.148362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.160952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.162732 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-ae756af8-93ac-4241-a911-8a70b190ab86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:11.180959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:11.182045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:11.183150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:11.183150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:11.183150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:11.183150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:11.189348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 532/1073] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:11.191019 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.216378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712d58d-807f-4706-b9b4-805d8e42ef7a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.216656 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712d58d-807f-4706-b9b4-805d8e42ef7a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.216929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712d58d-807f-4706-b9b4-805d8e42ef7a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.218358 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 533/1074] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:36:11.223416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.223913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.225122 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.238224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:11.244944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:11.246176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:11.299469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1eb0-47b8-48dd-98fd-7b9d20c3e9d2 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.317372 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 534/1075] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:36:11.336822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31179-a348-4beb-805d-65546938189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.337629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31179-a348-4beb-805d-65546938189c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.337912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31179-a348-4beb-805d-65546938189c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.338175 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31179-a348-4beb-805d-65546938189c None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.338895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e31179-a348-4beb-805d-65546938189c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:11.339192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e31179-a348-4beb-805d-65546938189c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.340925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e31179-a348-4beb-805d-65546938189c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.343985 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.356188 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 542/1076] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:11.362378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31179-a348-4beb-805d-65546938189c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.411203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:11.447378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54445c-e120-45dc-b939-4d6fe2b2bedf None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.466355 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31179-a348-4beb-805d-65546938189c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.486613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.487539 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.487824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.488105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.491044 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:11.501913 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:11.510233 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:11.510484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:11.511410 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:11.512253 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:11.512881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.512881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.512881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.513158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 535/1077] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:11.517866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.520265 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.521799 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:36:11.536405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:11.537463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:11.538374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 543/1078] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:11.543305 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.622026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a8c12-1c5a-43dc-8cbc-59669061ff39 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.641784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee69c7e-8363-41c9-8c1d-d54d1f13cfe6 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.649693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:11.659048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.659737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.660018 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.660290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.661131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:11.661469 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.662926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.671973 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 536/1079] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:11.677862 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.680129 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.680811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.681171 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.681505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.682301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:11.682660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.684421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.697274 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 544/1080] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:11.702559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.740848 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:11.785433 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30fa42-598b-4789-8edc-c0675a8a4e27 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.807586 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c47a8-f97c-4b6d-b1df-ab71f1360c2c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.818962 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.819849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.820126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.820396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] PATH_INFO: `/v3/users/2e8810a880414be7b8c0b6b1ec7202b4` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.822000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] RBAC: Authorizing `identity:get_user(user_id=2e8810a880414be7b8c0b6b1ec7202b4)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.823418 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.825096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 537/1081] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/users/2e8810a880414be7b8c0b6b1ec7202b4 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:11.830736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:11.841081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.842098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.842369 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.842617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/f5c6cd73a1234dde953e73fedc6440ee/roles/d2408e7e1e3241d6ae009ae98520c593` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.844957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=f5c6cd73a1234dde953e73fedc6440ee, role_id=d2408e7e1e3241d6ae009ae98520c593)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.846421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.859834 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 545/1082] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:36:11 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/f5c6cd73a1234dde953e73fedc6440ee/roles/d2408e7e1e3241d6ae009ae98520c593 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:36:11.941692 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cada64-8b4d-4da1-af17-ebc4332f6313 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:11.972743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:11.983020 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300ea09-f19c-4a2c-aa50-e72e5b1e8f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:11.983912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300ea09-f19c-4a2c-aa50-e72e5b1e8f8a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:11.984194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300ea09-f19c-4a2c-aa50-e72e5b1e8f8a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:11.984470 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300ea09-f19c-4a2c-aa50-e72e5b1e8f8a None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:11.985752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300ea09-f19c-4a2c-aa50-e72e5b1e8f8a None admin] RBAC: Authorizing `identity:get_project(project_id=01c4756c2b02470786b8d715f60de387)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:11.987220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300ea09-f19c-4a2c-aa50-e72e5b1e8f8a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:11.989100 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 538/1083] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:11 2022] GET /identity/v3/projects/01c4756c2b02470786b8d715f60de387 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:11.995663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:12.028863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:12.104973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732dacf-6cac-4e5b-bd32-c43fd9165ed9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:12.138807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2c252-a1b0-4975-8aa8-dacb41b750b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:12.139704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2c252-a1b0-4975-8aa8-dacb41b750b5 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:12.139973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2c252-a1b0-4975-8aa8-dacb41b750b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:12.140238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2c252-a1b0-4975-8aa8-dacb41b750b5 None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387/users/2e8810a880414be7b8c0b6b1ec7202b4/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:12.142254 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2c252-a1b0-4975-8aa8-dacb41b750b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=01c4756c2b02470786b8d715f60de387, user_id=2e8810a880414be7b8c0b6b1ec7202b4, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:12.143690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2c252-a1b0-4975-8aa8-dacb41b750b5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:12.158899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 546/1084] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:36:11 2022] PUT /identity/v3/projects/01c4756c2b02470786b8d715f60de387/users/2e8810a880414be7b8c0b6b1ec7202b4/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:36:14.071824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a153-38d0-4f50-b6ed-c4c2dfbee70d None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.072406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a153-38d0-4f50-b6ed-c4c2dfbee70d None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.072406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a153-38d0-4f50-b6ed-c4c2dfbee70d None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.073768 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 539/1085] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:14.079358 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af63d5-d8cd-47aa-b3f7-20091dabba49 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.079784 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af63d5-d8cd-47aa-b3f7-20091dabba49 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.079971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af63d5-d8cd-47aa-b3f7-20091dabba49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.097298 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-42af63d5-d8cd-47aa-b3f7-20091dabba49 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:14.098561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-42af63d5-d8cd-47aa-b3f7-20091dabba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:14.169088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 547/1086] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:14.193502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a153-38d0-4f50-b6ed-c4c2dfbee70d None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.298457 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a153-38d0-4f50-b6ed-c4c2dfbee70d None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.333389 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.334176 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.334453 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.334716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.345439 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:14.348066 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669bc05-8616-4037-bd44-ccb06f9cb1c4 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.348352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669bc05-8616-4037-bd44-ccb06f9cb1c4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.348626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669bc05-8616-4037-bd44-ccb06f9cb1c4 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.350068 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 548/1087] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:14.353401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:14.354327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:14.355131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 540/1088] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:14.355805 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.355805 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.355805 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.360502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.372557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:14.373763 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:14.385105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:14.420358 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:14.442411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 549/1089] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:14.464757 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff179b-7394-49e7-9fcd-362cd731a6a8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.465789 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.502459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.503149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.503407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.503680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.504497 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:14.504817 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:14.506236 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:14.515849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 541/1090] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:14.521531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.570426 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababccda-2bd7-43bc-b2a5-a82bd310f9c5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.604587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.605429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.605714 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.605987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.616007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:14.621668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762f63-6da7-4219-af86-3d34a21050cd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:14.623672 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:14.624637 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:14.625723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 550/1091] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:14.631405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.639354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:14.640660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:14.658540 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.659351 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.659625 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.659899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.671376 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:14.672878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:14.690065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:14.691024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:14.692125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:14.692125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:14.692125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:14.692125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:14.692125 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 542/1092] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:14.698328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.734941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b45b9-6c9f-4fc8-9732-4647b6afc1b9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.768847 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.769591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.769874 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.770239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.771000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:14.771301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:14.772711 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:14.783338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 551/1093] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:14.789425 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.802795 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cc832-0bb0-44de-b9cc-bca03c339219 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.835985 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.836686 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.836935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.837207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.837964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:14.838268 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:14.839641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:14.851413 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 543/1094] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:14.853321 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:14.857220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.886058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:14.887790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548013-cbaf-4faa-8cf7-4a63239f1b25 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.924598 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.925542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.925829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.926100 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] PATH_INFO: `/v3/users/2e8810a880414be7b8c0b6b1ec7202b4` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:14.927370 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] RBAC: Authorizing `identity:get_user(user_id=2e8810a880414be7b8c0b6b1ec7202b4)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:14.928832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:14.930557 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 552/1095] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/users/2e8810a880414be7b8c0b6b1ec7202b4 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:14.936367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:14.963675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afdb98-6299-439a-90cf-d652784107fa None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:14.997105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:14.998026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:14.998303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:14.998572 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.005870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.007352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:36:15.021743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:15.022669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:15.024130 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 544/1096] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:15.028988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.037899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b69090-2af0-4d31-a7e8-75dbc9da7a89 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.072073 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.072727 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.073006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.073386 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.076946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.114147 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.127386 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f2238-b193-489b-aad8-2a132622ff06 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.143152 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.144631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.162826 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.163528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.163791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.164081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.164921 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:15.165224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.166711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.184173 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 545/1097] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:15.190095 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.246380 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 553/1098] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:15.252211 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.286651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.294634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd565032-41e5-4ebe-a880-1938cfdf8f32 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.329877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.330537 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.330816 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.331091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.346829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.352470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b66f9f-ffec-4af2-9f6f-802baef0cad1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.385961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.386764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.387046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.387306 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.388538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] RBAC: Authorizing `identity:get_project(project_id=01c4756c2b02470786b8d715f60de387)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.390051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.396923 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 554/1099] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/projects/01c4756c2b02470786b8d715f60de387 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:15.397912 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.399858 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.401348 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.500099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f80430-f860-4d4a-ae6b-0b5d3550f0f4 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.504630 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.512352 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 546/1100] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:36:15.515894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.543151 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.543843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.544120 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.544405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.545250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=01c4756c2b02470786b8d715f60de387, user.id=2e8810a880414be7b8c0b6b1ec7202b4) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:15.545581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.547029 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.552892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 555/1101] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/role_assignments?user.id=2e8810a880414be7b8c0b6b1ec7202b4&scope.project.id=01c4756c2b02470786b8d715f60de387&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:36:15.563710 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.625579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10b7ea-4e9d-4c75-8d3b-3ae699c93bb7 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.664716 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2628895-7200-419b-8d42-0367abea257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.665478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2628895-7200-419b-8d42-0367abea257b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2628895-7200-419b-8d42-0367abea257b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.666025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2628895-7200-419b-8d42-0367abea257b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.678482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2628895-7200-419b-8d42-0367abea257b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.679975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2628895-7200-419b-8d42-0367abea257b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-f2628895-7200-419b-8d42-0367abea257b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:15.700155 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:15.701361 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:15.707645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:36:15.707645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:15.707645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:36:15.707645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:15.707645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 547/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:36:15.707954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.811108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966129cd-3d0c-4798-a119-6590fbd4e618 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:15.849113 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:15.849809 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:15.850091 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:15.850360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:15.851088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:15.851449 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:15.853201 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:15.864936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 556/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:15.870724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2628895-7200-419b-8d42-0367abea257b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:15.970701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:15.974938 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2628895-7200-419b-8d42-0367abea257b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:16.011300 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:16.012105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:16.012380 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:16.012647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:16.019472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:16.021026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:36:16.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:16.036087 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:16.037200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 548/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:16.042408 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:16.074197 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:16.146639 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db79c6-5ad1-46a3-9415-636071785264 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:16.186529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:16.187200 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:16.187455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:16.187748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:16.188530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:16.188810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:16.190439 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c598f-b812-4317-9f53-a12a1dfd38bb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:16.202843 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 557/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:16.209297 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:16.313856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3df1-2dc8-4a78-be76-48a50fe3b081 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:16.348842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:16.349528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:16.349803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:16.350075 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:16.350904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] RBAC: Adding query filter params (role.id=d2408e7e1e3241d6ae009ae98520c593, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=f5c6cd73a1234dde953e73fedc6440ee) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:16.351210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:16.352653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278da62b-bd0c-472c-b033-7646ac0c4580 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:16.358670 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 549/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:16 2022] GET /identity/v3/role_assignments?user.id=f5c6cd73a1234dde953e73fedc6440ee&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=d2408e7e1e3241d6ae009ae98520c593 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:17.805683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724316a6-6d7b-4aa7-9f74-ed4f4d0f309a None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:17.806214 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724316a6-6d7b-4aa7-9f74-ed4f4d0f309a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:17.806214 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724316a6-6d7b-4aa7-9f74-ed4f4d0f309a None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:17.807563 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 558/1107] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:17.814111 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:17.814904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:17.815050 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:17.840476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:17.841728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:17.861091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:17.932941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 550/1108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:17.958699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724316a6-6d7b-4aa7-9f74-ed4f4d0f309a None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:18.054666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:18.074991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724316a6-6d7b-4aa7-9f74-ed4f4d0f309a None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:18.115116 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a8ff3-ac43-4d08-a881-0e34c2ab4089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:18.115899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a8ff3-ac43-4d08-a881-0e34c2ab4089 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:18.116177 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a8ff3-ac43-4d08-a881-0e34c2ab4089 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:18.116445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a8ff3-ac43-4d08-a881-0e34c2ab4089 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:18.117810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a8ff3-ac43-4d08-a881-0e34c2ab4089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:18.119281 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a8ff3-ac43-4d08-a881-0e34c2ab4089 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:18.120869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 559/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:18.127024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:18.230475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6e9f-ac6e-4a19-894e-79825f5d56d7 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:18.272927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbded3ae-c373-4637-95b1-d9cf2ae0afbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:18.274007 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbded3ae-c373-4637-95b1-d9cf2ae0afbd None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:18.274290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbded3ae-c373-4637-95b1-d9cf2ae0afbd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:18.274563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbded3ae-c373-4637-95b1-d9cf2ae0afbd None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:18.275280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbded3ae-c373-4637-95b1-d9cf2ae0afbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:18.276736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbded3ae-c373-4637-95b1-d9cf2ae0afbd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:18.297168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 551/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:36:18 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:20.543749 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23a6f3-6ddf-42cc-b0c8-c435ca7eec14 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:20.544252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23a6f3-6ddf-42cc-b0c8-c435ca7eec14 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:20.544252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23a6f3-6ddf-42cc-b0c8-c435ca7eec14 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:20.546545 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 560/1111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:20.555349 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:20.555772 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:20.555953 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:20.578842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:20.580968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:20.583353 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:20.656964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 552/1112] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:20.694884 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23a6f3-6ddf-42cc-b0c8-c435ca7eec14 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:20.810327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23a6f3-6ddf-42cc-b0c8-c435ca7eec14 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:20.846085 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:20.846944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:20.847222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:20.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] PATH_INFO: `/v3/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:20.849114 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] RBAC: Authorizing `identity:get_role(role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:20.850558 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:20.851926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 561/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:20 2022] GET /identity/v3/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:20.858448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:20.960659 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa456de-f595-440e-ba2d-7ef9bd322788 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:20.998592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:20.999478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:20.999755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:21.000029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] PATH_INFO: `/v3/users/5a5f03d6b0914987855500d30a688751` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:21.007603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:21.015377 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5f03d6b0914987855500d30a688751)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:21.016832 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:21.018512 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 553/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:20 2022] GET /identity/v3/users/5a5f03d6b0914987855500d30a688751 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:21.025405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:21.054649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:21.137374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadc5a-3152-4de2-97d5-bdf33c0243a3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:21.172559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:21.173196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:21.173504 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:21.173777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:21.240510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:21.241999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:21.265958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:21.357567 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 562/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:21.366571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:21.484547 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012edd56-cdc0-4753-a323-6238e5ca57c9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:21.522443 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d3b19-23b8-4f53-b59f-d89ecb3da36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:21.523346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d3b19-23b8-4f53-b59f-d89ecb3da36d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:21.523982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d3b19-23b8-4f53-b59f-d89ecb3da36d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:21.524248 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d3b19-23b8-4f53-b59f-d89ecb3da36d None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:21.525515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d3b19-23b8-4f53-b59f-d89ecb3da36d None admin] RBAC: Authorizing `identity:get_project(project_id=01c4756c2b02470786b8d715f60de387)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:21.526959 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d3b19-23b8-4f53-b59f-d89ecb3da36d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:21.528700 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 554/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:21 2022] GET /identity/v3/projects/01c4756c2b02470786b8d715f60de387 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:21.535057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:21.637710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da889-1b5f-4b89-86f6-32893c0523d7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:21.661873 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:21.674344 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:21.674993 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:21.675271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:21.675537 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:21.676466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] RBAC: Adding query filter params (role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=01c4756c2b02470786b8d715f60de387, user.id=5a5f03d6b0914987855500d30a688751) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:21.676764 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:21.678234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e476303-ca77-483c-84f9-200e298901cd None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:21.683814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 563/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:21 2022] GET /identity/v3/role_assignments?user.id=5a5f03d6b0914987855500d30a688751&scope.project.id=01c4756c2b02470786b8d715f60de387&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:23.871238 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da62026-97c1-4701-ae79-05174385f0d6 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:23.872102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da62026-97c1-4701-ae79-05174385f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:23.872102 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da62026-97c1-4701-ae79-05174385f0d6 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:23.874335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 555/1118] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:23.880922 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:23.881682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:23.881832 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:23.904887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:23.906115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:23.984181 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 564/1119] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:24.009342 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da62026-97c1-4701-ae79-05174385f0d6 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:24.088291 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:24.132793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da62026-97c1-4701-ae79-05174385f0d6 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:24.171666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:24.172550 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:24.172831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:24.173105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] PATH_INFO: `/v3/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:24.174680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] RBAC: Authorizing `identity:get_role(role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:24.176055 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:24.178031 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 556/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:24 2022] GET /identity/v3/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:24.185702 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:24.301898 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864eeba-a28c-49fc-9b27-a8f78c9fe43f None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:24.320368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:24.338971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:24.339856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:24.340132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:24.340399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] PATH_INFO: `/v3/users/5a5f03d6b0914987855500d30a688751` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:24.341682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5f03d6b0914987855500d30a688751)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:24.343152 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:24.344815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 565/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:24 2022] GET /identity/v3/users/5a5f03d6b0914987855500d30a688751 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:24.351614 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:24.466531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb63a6-b256-4a77-b8df-71f4990835ee None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:24.496011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:24.501876 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feafe369-dbc7-42db-9cf6-19074751e0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:24.502749 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafe369-dbc7-42db-9cf6-19074751e0ed None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:24.503024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafe369-dbc7-42db-9cf6-19074751e0ed None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:24.503293 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafe369-dbc7-42db-9cf6-19074751e0ed None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:24.504564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feafe369-dbc7-42db-9cf6-19074751e0ed None admin] RBAC: Authorizing `identity:get_project(project_id=01c4756c2b02470786b8d715f60de387)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:24.506002 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feafe369-dbc7-42db-9cf6-19074751e0ed None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:24.507709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 557/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:24 2022] GET /identity/v3/projects/01c4756c2b02470786b8d715f60de387 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:24.513831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:24.617075 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87da0e8-4c11-41a3-baa7-ff98bac85a22 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:24.653332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c95b3-1e4a-47c2-bd8c-63af976cad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:24.654219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c95b3-1e4a-47c2-bd8c-63af976cad34 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:24.654497 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c95b3-1e4a-47c2-bd8c-63af976cad34 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:24.654767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c95b3-1e4a-47c2-bd8c-63af976cad34 None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387/users/5a5f03d6b0914987855500d30a688751/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:24.657437 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c95b3-1e4a-47c2-bd8c-63af976cad34 None admin] RBAC: Authorizing `identity:create_grant(project_id=01c4756c2b02470786b8d715f60de387, user_id=5a5f03d6b0914987855500d30a688751, role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:24.658883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c95b3-1e4a-47c2-bd8c-63af976cad34 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:24.676663 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 566/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:36:24 2022] PUT /identity/v3/projects/01c4756c2b02470786b8d715f60de387/users/5a5f03d6b0914987855500d30a688751/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:36:26.859907 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1141e-4ad0-4ec8-b9ca-e4287efb16e5 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:26.860757 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1141e-4ad0-4ec8-b9ca-e4287efb16e5 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:26.860757 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1141e-4ad0-4ec8-b9ca-e4287efb16e5 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:26.863012 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 558/1124] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:26.871723 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:26.872135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:26.872340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:26.897892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:26.899425 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:26.945353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:26.975711 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 567/1125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:27.004503 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1141e-4ad0-4ec8-b9ca-e4287efb16e5 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:27.118982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1141e-4ad0-4ec8-b9ca-e4287efb16e5 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:27.154866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:27.155725 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:27.156000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:27.156276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] PATH_INFO: `/v3/roles/f62ac7e18b8d47c5a793012df739fa9b` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:27.157916 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] RBAC: Authorizing `identity:get_role(role_id=f62ac7e18b8d47c5a793012df739fa9b)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:27.159294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:27.160772 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 559/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:27 2022] GET /identity/v3/roles/f62ac7e18b8d47c5a793012df739fa9b => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:27.166976 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:27.271867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51c116-80d9-4588-91d2-e7f5d53ebeaf None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:27.308390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:27.309287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:27.309547 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:27.309817 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] PATH_INFO: `/v3/users/5a5f03d6b0914987855500d30a688751` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:27.311088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] RBAC: Authorizing `identity:get_user(user_id=5a5f03d6b0914987855500d30a688751)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:27.312538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:27.315658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 568/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:27 2022] GET /identity/v3/users/5a5f03d6b0914987855500d30a688751 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:27.325562 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:27.402962 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:27.441587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c90e0-72ac-405f-b55c-ade6bfc7bcd8 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:27.478568 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:27.479215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:27.479492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:27.479760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:27.550095 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:27.551605 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:27.619885 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:27.657489 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 560/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:27.663461 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:27.710003 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:27.780241 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18f60c-06a4-4e9b-ba4d-e9b95ef86fa3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:27.816685 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ece61f-2ccc-4241-ade7-aba6aaa7b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:27.817574 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ece61f-2ccc-4241-ade7-aba6aaa7b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:27.817858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ece61f-2ccc-4241-ade7-aba6aaa7b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:27.818130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ece61f-2ccc-4241-ade7-aba6aaa7b6f2 None admin] PATH_INFO: `/v3/projects/01c4756c2b02470786b8d715f60de387` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:27.819448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ece61f-2ccc-4241-ade7-aba6aaa7b6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=01c4756c2b02470786b8d715f60de387)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:27.820901 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ece61f-2ccc-4241-ade7-aba6aaa7b6f2 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:27.822632 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 569/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:27 2022] GET /identity/v3/projects/01c4756c2b02470786b8d715f60de387 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:27.832232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:27.934666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0f360-9477-4109-a3d1-ab0ccc6c8ad6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:27.979098 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:27.979756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:27.980039 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:27.980316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:27.981144 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] RBAC: Adding query filter params (role.id=f62ac7e18b8d47c5a793012df739fa9b, scope.project.id=01c4756c2b02470786b8d715f60de387, user.id=5a5f03d6b0914987855500d30a688751) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:27.981502 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:27.982944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857a981-c420-4f0b-8ede-d9474165fceb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:27.989395 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 561/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:27 2022] GET /identity/v3/role_assignments?user.id=5a5f03d6b0914987855500d30a688751&scope.project.id=01c4756c2b02470786b8d715f60de387&role.id=f62ac7e18b8d47c5a793012df739fa9b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:30.204524 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38297d3a-aab9-4ae5-a558-ab5b9d50e398 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:30.205104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38297d3a-aab9-4ae5-a558-ab5b9d50e398 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:30.205104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38297d3a-aab9-4ae5-a558-ab5b9d50e398 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:30.206469 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 570/1131] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:30.212064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:30.212477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:30.212594 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:30.231392 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:30.232555 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:30.261915 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:30.314932 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 562/1132] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:30.351883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38297d3a-aab9-4ae5-a558-ab5b9d50e398 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:30.483891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38297d3a-aab9-4ae5-a558-ab5b9d50e398 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:30.497370 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:30.524942 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d67b5d-8587-4a22-98ce-9d6dfcc7a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:30.525791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d67b5d-8587-4a22-98ce-9d6dfcc7a610 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:30.526076 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d67b5d-8587-4a22-98ce-9d6dfcc7a610 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:30.526347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d67b5d-8587-4a22-98ce-9d6dfcc7a610 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:30.527604 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d67b5d-8587-4a22-98ce-9d6dfcc7a610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:30.529063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d67b5d-8587-4a22-98ce-9d6dfcc7a610 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:30.530724 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 571/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:30.536739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:30.646053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea48ae-a187-4104-bb80-93a125c6ad5c None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:30.679010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d7e50-1272-4bd1-9a99-fd56122d4c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:30.679663 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7e50-1272-4bd1-9a99-fd56122d4c0b None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:30.679946 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7e50-1272-4bd1-9a99-fd56122d4c0b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:30.680219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7e50-1272-4bd1-9a99-fd56122d4c0b None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:30.680941 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d7e50-1272-4bd1-9a99-fd56122d4c0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:30.682430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d7e50-1272-4bd1-9a99-fd56122d4c0b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:30.713246 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 563/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:36:30 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:36:32.949266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c7a7-ddce-424c-b82a-70541a2e9ab0 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:32.949852 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c7a7-ddce-424c-b82a-70541a2e9ab0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:32.949852 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c7a7-ddce-424c-b82a-70541a2e9ab0 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:32.951264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 572/1135] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:32.957945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:32.958673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:32.958823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:32.980848 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:32.982063 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:32.992008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:33.055984 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 564/1136] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:33.080677 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1c7a7-ddce-424c-b82a-70541a2e9ab0 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:33.190992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1c7a7-ddce-424c-b82a-70541a2e9ab0 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:33.228315 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afb11-fafa-4736-9067-4d5327a20b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:33.229036 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afb11-fafa-4736-9067-4d5327a20b12 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:33.229307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afb11-fafa-4736-9067-4d5327a20b12 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:33.229616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afb11-fafa-4736-9067-4d5327a20b12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:33.230932 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23afb11-fafa-4736-9067-4d5327a20b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:33.232391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23afb11-fafa-4736-9067-4d5327a20b12 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:33.234038 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 573/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:33.239903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:33.344646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195909a-96e7-4c9d-b859-50a3688d38aa None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:33.380207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c94f0c-37d8-4090-8cd4-a805a526152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:33.380827 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c94f0c-37d8-4090-8cd4-a805a526152f None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:33.381107 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c94f0c-37d8-4090-8cd4-a805a526152f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:33.381417 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c94f0c-37d8-4090-8cd4-a805a526152f None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:33.382123 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c94f0c-37d8-4090-8cd4-a805a526152f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:33.383547 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c94f0c-37d8-4090-8cd4-a805a526152f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:33.403961 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:33.412500 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 565/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:36:33 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:35.664986 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ab884-7905-4dca-95f0-a67d1fc6c2b5 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:35.665879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ab884-7905-4dca-95f0-a67d1fc6c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:35.665879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ab884-7905-4dca-95f0-a67d1fc6c2b5 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:35.668080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 574/1139] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:35.676126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:35.676541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:35.676648 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:35.695424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:35.696609 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:35.768775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 566/1140] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:35.805747 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ab884-7905-4dca-95f0-a67d1fc6c2b5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:35.869963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:35.932367 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ab884-7905-4dca-95f0-a67d1fc6c2b5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:35.969959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:35.970760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:35.971041 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:35.971313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:35.984012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:35.991842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:35.992965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:35.993959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 575/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:35.999283 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:36.104898 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559523a-5cd2-48f2-badb-0c111c8fae78 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:36.134511 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:36.141481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:36.142153 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:36.142433 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:36.142703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:36.143429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:36.143736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:36.145219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:36.154045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 567/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:36.160213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:36.278168 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c724f-621c-4fd1-b183-f9abe8319187 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:36.287001 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:36.325978 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:36.326950 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:36.327235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:36.327510 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] PATH_INFO: `/v3/users/3fd595d89fb9490fb7fe72848dd9e1c4` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:36.341350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] RBAC: Authorizing `identity:get_user(user_id=3fd595d89fb9490fb7fe72848dd9e1c4)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:36.342822 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:36.344495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 576/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:36 2022] GET /identity/v3/users/3fd595d89fb9490fb7fe72848dd9e1c4 => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:36.350691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:36.455210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146652e-e937-44f2-9a8e-218107f71f44 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:36.489854 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:36.490484 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:36.490760 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:36.491022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:36.552751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:36.564192 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:36.566512 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:36.674005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 568/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:36.680743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:36.795609 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694669be-d70a-476c-ae76-87a7866bffb8 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:36.832681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee952b-256c-4e0b-922e-33b58185c2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:36.833576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee952b-256c-4e0b-922e-33b58185c2b3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:36.833853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee952b-256c-4e0b-922e-33b58185c2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:36.834116 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee952b-256c-4e0b-922e-33b58185c2b3 None admin] PATH_INFO: `/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:36.835353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee952b-256c-4e0b-922e-33b58185c2b3 None admin] RBAC: Authorizing `identity:get_project(project_id=3c0ad8d8c4de4e85bd3738e4c6b11d0f)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:36.836797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee952b-256c-4e0b-922e-33b58185c2b3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:36.838520 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 577/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:36 2022] GET /identity/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:36.844872 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:36.940340 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:36.950815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d2d13-9420-4056-ac94-ef56cf9ba061 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:36.988397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:36.989053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:36.989375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:36.989649 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:36.990475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=3c0ad8d8c4de4e85bd3738e4c6b11d0f, user.id=3fd595d89fb9490fb7fe72848dd9e1c4) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:36.990781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:36.992216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29624450-2f8f-4509-9022-1846f0157526 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:36.997918 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 569/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:36 2022] GET /identity/v3/role_assignments?user.id=3fd595d89fb9490fb7fe72848dd9e1c4&scope.project.id=3c0ad8d8c4de4e85bd3738e4c6b11d0f&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:39.170507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3350f2-ee9a-4082-9143-713e02a555b7 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:39.171343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3350f2-ee9a-4082-9143-713e02a555b7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:39.171343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3350f2-ee9a-4082-9143-713e02a555b7 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:39.177289 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 578/1147] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:39.188620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:39.189149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:39.189673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:39.219252 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:39.221096 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:39.296172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 570/1148] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:39.319967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3350f2-ee9a-4082-9143-713e02a555b7 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:39.369103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:39.438519 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3350f2-ee9a-4082-9143-713e02a555b7 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:39.486868 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:39.487666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:39.487963 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:39.488237 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:39.500332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:39.508054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:39.509108 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:39.509970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 579/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:39.515447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:39.610295 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:39.626657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217acbf-5e07-47eb-814c-b570b0657438 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:39.662578 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:39.663261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:39.663541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:39.663816 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:39.664556 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:39.664860 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:39.666273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:39.675394 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 571/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:39.682703 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:39.800227 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:39.808608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399917f1-5a51-4e21-a2a0-1270155e8f32 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:39.847249 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:39.848162 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:39.848441 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:39.848715 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] PATH_INFO: `/v3/users/3fd595d89fb9490fb7fe72848dd9e1c4` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:39.850089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] RBAC: Authorizing `identity:get_user(user_id=3fd595d89fb9490fb7fe72848dd9e1c4)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:39.851546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:39.853221 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 580/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:39 2022] GET /identity/v3/users/3fd595d89fb9490fb7fe72848dd9e1c4 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:39.859360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:39.970979 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e08752-d094-4284-9299-5d9aaa4198a9 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:40.020680 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df283fac-ad74-4b3d-95ae-18f1e17d4c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:40.021588 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df283fac-ad74-4b3d-95ae-18f1e17d4c09 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:40.021874 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df283fac-ad74-4b3d-95ae-18f1e17d4c09 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:40.022149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df283fac-ad74-4b3d-95ae-18f1e17d4c09 None admin] PATH_INFO: `/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:40.023799 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df283fac-ad74-4b3d-95ae-18f1e17d4c09 None admin] RBAC: Authorizing `identity:get_project(project_id=3c0ad8d8c4de4e85bd3738e4c6b11d0f)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:40.025229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df283fac-ad74-4b3d-95ae-18f1e17d4c09 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:40.026989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 572/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:39 2022] GET /identity/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:40.033084 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:40.141430 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2817ed0-0ec0-41ca-baa1-9458f2062788 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:40.177398 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c98e8e-7920-44c4-b5e3-c7fcc2ea3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:40.178295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c98e8e-7920-44c4-b5e3-c7fcc2ea3d14 None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:40.178569 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c98e8e-7920-44c4-b5e3-c7fcc2ea3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:40.178842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c98e8e-7920-44c4-b5e3-c7fcc2ea3d14 None admin] PATH_INFO: `/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f/users/3fd595d89fb9490fb7fe72848dd9e1c4/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:40.180881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c98e8e-7920-44c4-b5e3-c7fcc2ea3d14 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c0ad8d8c4de4e85bd3738e4c6b11d0f, user_id=3fd595d89fb9490fb7fe72848dd9e1c4, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:40.182399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c98e8e-7920-44c4-b5e3-c7fcc2ea3d14 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:40.199338 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 581/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:36:40 2022] PUT /identity/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f/users/3fd595d89fb9490fb7fe72848dd9e1c4/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:36:42.388030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfdb9c-77d4-4080-b0cd-b251ade99ee8 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:42.388978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfdb9c-77d4-4080-b0cd-b251ade99ee8 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:42.388978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfdb9c-77d4-4080-b0cd-b251ade99ee8 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:42.391122 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 573/1154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:42.397651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:42.398738 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:42.398875 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:42.420589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:42.421819 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:42.439212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:42.493222 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 582/1155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:42.517898 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfdb9c-77d4-4080-b0cd-b251ade99ee8 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:42.549945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:42.640178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfdb9c-77d4-4080-b0cd-b251ade99ee8 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:42.675899 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:42.676688 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:42.676971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:42.677273 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:42.688523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:42.696158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:42.697280 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:42.698535 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 574/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:42.706874 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:42.813942 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80278726-b9e7-4bee-acdf-b6d6fb758049 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:42.847331 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:42.847998 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:42.848279 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:42.848549 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:42.849290 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:42.849590 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:42.850981 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:42.858899 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 583/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:42.865796 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:42.963757 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:42.990635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f060e-d77a-4e07-96f2-60eea4474f48 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:43.025592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:43.026452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:43.026728 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:43.026993 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] PATH_INFO: `/v3/users/3fd595d89fb9490fb7fe72848dd9e1c4` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:43.028304 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] RBAC: Authorizing `identity:get_user(user_id=3fd595d89fb9490fb7fe72848dd9e1c4)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:43.029768 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:43.031399 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 575/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:42 2022] GET /identity/v3/users/3fd595d89fb9490fb7fe72848dd9e1c4 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:43.037538 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:43.055429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:43.143773 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc4236-75f4-4c93-9744-1d55b3b5fe77 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:43.179698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:43.180337 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:43.180615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:43.180887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:43.249992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:43.251391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:43.265837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:43.352476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 584/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:43.359233 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:43.463507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305288a9-6fa5-4210-a41c-ec1ef845f33d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:43.502571 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5ed23-107b-4c83-9833-871bb735c57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:43.503374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5ed23-107b-4c83-9833-871bb735c57d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:43.503657 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5ed23-107b-4c83-9833-871bb735c57d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:43.503925 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5ed23-107b-4c83-9833-871bb735c57d None admin] PATH_INFO: `/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:43.505227 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5ed23-107b-4c83-9833-871bb735c57d None admin] RBAC: Authorizing `identity:get_project(project_id=3c0ad8d8c4de4e85bd3738e4c6b11d0f)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:43.507549 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5ed23-107b-4c83-9833-871bb735c57d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:43.509906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 576/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:43 2022] GET /identity/v3/projects/3c0ad8d8c4de4e85bd3738e4c6b11d0f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:43.517103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:43.627965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4012dc-c061-4bc1-b054-da1746b891a3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:43.643744 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:43.665838 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:43.666498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:43.666781 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:43.667053 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:43.667883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=3c0ad8d8c4de4e85bd3738e4c6b11d0f, user.id=3fd595d89fb9490fb7fe72848dd9e1c4) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:43.668196 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:43.669677 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2892b9-13f5-4781-afee-b9db9e4e6647 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:43.675754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 585/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:43 2022] GET /identity/v3/role_assignments?user.id=3fd595d89fb9490fb7fe72848dd9e1c4&scope.project.id=3c0ad8d8c4de4e85bd3738e4c6b11d0f&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:45.894856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b71d9-8d3a-4e50-876e-c8bc72166c34 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:45.895764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b71d9-8d3a-4e50-876e-c8bc72166c34 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:45.895764 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b71d9-8d3a-4e50-876e-c8bc72166c34 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:45.897969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 577/1162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:45.903841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:45.904283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:45.904379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:45.922071 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:45.923257 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:45.994602 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 586/1163] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:46.018061 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b71d9-8d3a-4e50-876e-c8bc72166c34 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:46.041333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:46.134308 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b71d9-8d3a-4e50-876e-c8bc72166c34 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:46.169475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:46.170045 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:46.170317 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:46.170581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:46.238022 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:46.239467 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:46.252972 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:46.339656 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 578/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:36:46.349311 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:46.457807 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036c303-1191-45fb-bd68-e29ce032b44f None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:46.466894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:46.498010 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:46.498826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:46.499106 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:46.499382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:46.506591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:46.508069 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-34ac29bb-1a00-443b-b88e-72df88177b35 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:46.522463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:46.523900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:36:46.524760 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 587/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:36:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:46.531837 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:46.642498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8bc85-620b-4273-8069-8fd5c53b41d3 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:46.656232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:46.680926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:46.681622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:46.681902 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:46.682170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:46.682886 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:46.683186 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:46.684578 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339006ea-ae59-4699-8e81-609b0a51adbb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:46.692730 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 579/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:36:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:48.892872 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d0acc-165b-4ac7-bf59-f97b088dfa06 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:48.893471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d0acc-165b-4ac7-bf59-f97b088dfa06 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:48.893471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d0acc-165b-4ac7-bf59-f97b088dfa06 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:48.894959 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 588/1167] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:48.901980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9b2f4-20d8-4edf-9ef0-36e29f866bfc None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:48.902701 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9b2f4-20d8-4edf-9ef0-36e29f866bfc None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:48.902857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9b2f4-20d8-4edf-9ef0-36e29f866bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:48.925298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-f3a9b2f4-20d8-4edf-9ef0-36e29f866bfc None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:48.926481 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-f3a9b2f4-20d8-4edf-9ef0-36e29f866bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:49.012818 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 580/1168] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:49.036515 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d0acc-165b-4ac7-bf59-f97b088dfa06 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:49.159102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d0acc-165b-4ac7-bf59-f97b088dfa06 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:49.194771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924c1a6-08ae-4c35-9ef5-c88b23e6c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:49.195429 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c1a6-08ae-4c35-9ef5-c88b23e6c284 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:49.195709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c1a6-08ae-4c35-9ef5-c88b23e6c284 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:49.195985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c1a6-08ae-4c35-9ef5-c88b23e6c284 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:49.196730 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c1a6-08ae-4c35-9ef5-c88b23e6c284 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:49.198159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c1a6-08ae-4c35-9ef5-c88b23e6c284 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:49.217218 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 589/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Thu May 19 14:36:49 2022] POST /identity/v3/domains => generated 267 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:51.436656 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebe2c-058a-4ca1-89e3-144f71130a48 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.437256 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebe2c-058a-4ca1-89e3-144f71130a48 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.437256 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebe2c-058a-4ca1-89e3-144f71130a48 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.439529 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 581/1170] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:51.445731 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.446172 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.446279 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.463790 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:51.464980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:51.489027 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:51.545713 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 590/1171] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:36:51.569497 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebe2c-058a-4ca1-89e3-144f71130a48 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:51.680775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebe2c-058a-4ca1-89e3-144f71130a48 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:51.688292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:51.719527 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0303-1015-4a7c-899f-13b17c28485f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:51.720387 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0303-1015-4a7c-899f-13b17c28485f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.720665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0303-1015-4a7c-899f-13b17c28485f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.720936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0303-1015-4a7c-899f-13b17c28485f None admin] PATH_INFO: `/v3/domains/187a5937c27a4dc4b33a6fe6f1326a19` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.737853 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af0303-1015-4a7c-899f-13b17c28485f None admin] RBAC: Authorizing `identity:get_domain(domain_id=187a5937c27a4dc4b33a6fe6f1326a19)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:51.739302 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af0303-1015-4a7c-899f-13b17c28485f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:51.740964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 582/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 14:36:51 2022] GET /identity/v3/domains/187a5937c27a4dc4b33a6fe6f1326a19 => generated 267 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:51.747018 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:51.857155 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cd740-3e4e-4f8a-a569-916665e4dde8 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:51.895981 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d95ae-2f6b-444b-ae18-4e13737b9bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:51.896642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d95ae-2f6b-444b-ae18-4e13737b9bae None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:51.896925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d95ae-2f6b-444b-ae18-4e13737b9bae None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:51.897200 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d95ae-2f6b-444b-ae18-4e13737b9bae None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:51.897944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d95ae-2f6b-444b-ae18-4e13737b9bae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:51.899743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d95ae-2f6b-444b-ae18-4e13737b9bae None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:51.928837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 591/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:36:51 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:54.152158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204a724-bec1-4602-95d0-7fb1c1e19a8b None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.153032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204a724-bec1-4602-95d0-7fb1c1e19a8b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.153032 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204a724-bec1-4602-95d0-7fb1c1e19a8b None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.155244 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 583/1174] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:54.162582 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.162996 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.163138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.181159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:54.182400 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:54.185195 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.254961 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 592/1175] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:36:54.291917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204a724-bec1-4602-95d0-7fb1c1e19a8b None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.403561 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204a724-bec1-4602-95d0-7fb1c1e19a8b None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.438963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad4bb3-f1fe-48f4-87a4-40cd9f368291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.439829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad4bb3-f1fe-48f4-87a4-40cd9f368291 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.440111 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad4bb3-f1fe-48f4-87a4-40cd9f368291 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.440385 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad4bb3-f1fe-48f4-87a4-40cd9f368291 None admin] PATH_INFO: `/v3/domains/187a5937c27a4dc4b33a6fe6f1326a19` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.441696 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad4bb3-f1fe-48f4-87a4-40cd9f368291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=187a5937c27a4dc4b33a6fe6f1326a19)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:54.443148 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad4bb3-f1fe-48f4-87a4-40cd9f368291 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:54.444766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 584/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 14:36:54 2022] GET /identity/v3/domains/187a5937c27a4dc4b33a6fe6f1326a19 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:54.451317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:54.560816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc59d5-ed3a-4c03-ace9-3740fc3015b3 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:54.595021 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4820b-45fb-45a4-be38-f7c4492183e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:54.595688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4820b-45fb-45a4-be38-f7c4492183e0 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:54.595964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4820b-45fb-45a4-be38-f7c4492183e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:54.596232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4820b-45fb-45a4-be38-f7c4492183e0 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:54.596936 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4820b-45fb-45a4-be38-f7c4492183e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:54.598426 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4820b-45fb-45a4-be38-f7c4492183e0 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:54.609761 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:54.625086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 593/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:36:54 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:36:56.864294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f932-e6c4-4569-8f51-80bc6d23d647 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:56.865138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f932-e6c4-4569-8f51-80bc6d23d647 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:56.865138 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f932-e6c4-4569-8f51-80bc6d23d647 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:56.867416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 585/1178] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:36:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:36:56.875545 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:56.875995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:56.876095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:56.894848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:36:56.896028 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:36:56.969482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 594/1179] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:36:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:36:56.993057 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f932-e6c4-4569-8f51-80bc6d23d647 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:57.033289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:57.105812 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f932-e6c4-4569-8f51-80bc6d23d647 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:57.140998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:57.141758 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:57.142035 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:57.142303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:57.154309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:36:57.164038 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:36:57.165094 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:36:57.166128 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 586/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:36:57.171498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:57.288705 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c166201-d9c1-4c57-8384-f3ae4c54e563 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:57.317977 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:57.327880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:57.328556 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:57.328837 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:57.329099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:57.329883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:57.330190 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:57.331629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:57.339983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 595/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:36:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:57.346206 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:57.442370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:57.449723 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21e3a6-320a-45c4-8dd6-0ffa597a065a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:57.486715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:57.487610 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:57.487883 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:57.488152 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] PATH_INFO: `/v3/users/d8e39ea28fd841399d0569d456d3cbd6` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:57.501803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] RBAC: Authorizing `identity:get_user(user_id=d8e39ea28fd841399d0569d456d3cbd6)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:57.503298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:57.504960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 587/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:36:57 2022] GET /identity/v3/users/d8e39ea28fd841399d0569d456d3cbd6 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:57.511024 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:57.626561 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcb6a1-c96d-422f-8175-08397cdf7685 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:57.670574 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:57.671212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:57.671491 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:57.671759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:57.730180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:57.744246 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:57.745748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:57.850476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 596/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:36:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:36:57.857416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:57.977782 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd5e4d-73e1-406f-8d5e-4e4c2ab6a61e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.016565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84ceb5-15d9-45c7-9eb5-2df6eaaed0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.017454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84ceb5-15d9-45c7-9eb5-2df6eaaed0b6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.017737 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84ceb5-15d9-45c7-9eb5-2df6eaaed0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.018008 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84ceb5-15d9-45c7-9eb5-2df6eaaed0b6 None admin] PATH_INFO: `/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.019303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84ceb5-15d9-45c7-9eb5-2df6eaaed0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=45939c9fb3c94fad9f38a7a8fecd6af2)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:58.020756 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84ceb5-15d9-45c7-9eb5-2df6eaaed0b6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:58.022530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 588/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:36:57 2022] GET /identity/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2 => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:36:58.029327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:36:58.120655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:36:58.137269 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751f157-796e-4776-9b33-8cefab4f3194 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:36:58.173656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:36:58.174296 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:36:58.174576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:36:58.174848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:36:58.175681 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=45939c9fb3c94fad9f38a7a8fecd6af2, user.id=d8e39ea28fd841399d0569d456d3cbd6) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:36:58.175988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:36:58.177465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26da7b7-5ffc-4521-a086-b867e2cdba40 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:36:58.183626 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 597/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:36:58 2022] GET /identity/v3/role_assignments?user.id=d8e39ea28fd841399d0569d456d3cbd6&scope.project.id=45939c9fb3c94fad9f38a7a8fecd6af2&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:00.335641 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301911f-a666-45be-b4e5-688f842300ae None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.336226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301911f-a666-45be-b4e5-688f842300ae None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.336226 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301911f-a666-45be-b4e5-688f842300ae None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.337597 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 589/1186] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:00.343951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.344666 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.344803 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.363372 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:00.364570 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:00.442142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 598/1187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:37:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:37:00.476884 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301911f-a666-45be-b4e5-688f842300ae None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:00.501576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:00.589028 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301911f-a666-45be-b4e5-688f842300ae None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:00.625823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:00.626616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.626898 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.627167 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.637681 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:00.645440 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:00.646472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:37:00.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 590/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:37:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:00.654077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:00.746300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:00.767796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd80c4-c391-4002-91c6-b509d113dad3 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:00.803153 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:00.803815 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.804093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.804360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.805102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:00.805437 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:00.806835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:00.814970 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 599/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:37:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:00.824655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:00.911373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:00.931640 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8378-43e0-4118-824d-d803c2b19835 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:00.969538 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:00.970426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:00.970695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:00.970967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] PATH_INFO: `/v3/users/d8e39ea28fd841399d0569d456d3cbd6` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:00.972330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] RBAC: Authorizing `identity:get_user(user_id=d8e39ea28fd841399d0569d456d3cbd6)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:00.973806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:00.975445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 591/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:37:00 2022] GET /identity/v3/users/d8e39ea28fd841399d0569d456d3cbd6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:00.982283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:01.093841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973691a1-924f-40b2-a18c-ce9528414200 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:01.128460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd3724-37eb-4350-b253-95d0ecca6cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:01.129307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3724-37eb-4350-b253-95d0ecca6cd7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:01.129603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3724-37eb-4350-b253-95d0ecca6cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:01.129831 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3724-37eb-4350-b253-95d0ecca6cd7 None admin] PATH_INFO: `/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:01.131164 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd3724-37eb-4350-b253-95d0ecca6cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=45939c9fb3c94fad9f38a7a8fecd6af2)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:01.132600 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd3724-37eb-4350-b253-95d0ecca6cd7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:01.134353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 600/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:37:00 2022] GET /identity/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:01.140335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:01.241562 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8814-b9ec-4497-96cc-f92badd7fa66 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:01.277364 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53123c-ed3e-469a-b1c0-eeb516d3c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:01.278249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53123c-ed3e-469a-b1c0-eeb516d3c55f None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:01.278531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53123c-ed3e-469a-b1c0-eeb516d3c55f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:01.278803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53123c-ed3e-469a-b1c0-eeb516d3c55f None admin] PATH_INFO: `/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2/users/d8e39ea28fd841399d0569d456d3cbd6/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:01.280894 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53123c-ed3e-469a-b1c0-eeb516d3c55f None admin] RBAC: Authorizing `identity:create_grant(project_id=45939c9fb3c94fad9f38a7a8fecd6af2, user_id=d8e39ea28fd841399d0569d456d3cbd6, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:01.282358 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53123c-ed3e-469a-b1c0-eeb516d3c55f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:01.300120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 592/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:37:01 2022] PUT /identity/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2/users/d8e39ea28fd841399d0569d456d3cbd6/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:37:03.504994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c7a24-d868-4cbd-a330-33817efa0df1 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:03.505560 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c7a24-d868-4cbd-a330-33817efa0df1 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:03.505669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c7a24-d868-4cbd-a330-33817efa0df1 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:03.507742 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 601/1193] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:37:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:37:03.516446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:03.516446 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:03.517146 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:03.536404 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:37:03.537699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:37:03.539653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:03.627044 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 593/1194] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:37:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:37:03.657698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c7a24-d868-4cbd-a330-33817efa0df1 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:03.673894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:03.769811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c7a24-d868-4cbd-a330-33817efa0df1 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:03.805258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:03.806080 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:03.806360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:03.806633 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:03.818353 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:37:03.826132 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:37:03.827161 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:37:03.828203 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 602/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:37:03.833774 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:03.941695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53b02d-6174-4a8d-87f2-b7f6907d5cf3 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:03.979285 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:03.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:03.980330 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:03.980638 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:03.981533 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:03.981872 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:03.983373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:03.988906 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:03.992851 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 594/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:37:03.999693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:04.084853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:04.116142 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd7fb9-f2a0-48ba-a722-108fd2d5dc6c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:04.155282 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:04.156145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:04.156427 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:04.156699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] PATH_INFO: `/v3/users/d8e39ea28fd841399d0569d456d3cbd6` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:04.158054 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] RBAC: Authorizing `identity:get_user(user_id=d8e39ea28fd841399d0569d456d3cbd6)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:04.159669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:04.161345 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 603/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:37:03 2022] GET /identity/v3/users/d8e39ea28fd841399d0569d456d3cbd6 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:04.167617 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:04.279251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316537fd-45b4-4556-8be8-363f5dd99156 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:04.315401 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:04.316064 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:04.316381 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:04.316691 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:04.386303 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:04.388905 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:04.391017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:04.503841 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 595/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:37:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:37:04.510134 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:04.617444 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cff85a-ace1-42d1-b83e-5d56b2456c6d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:04.658729 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8521760-7510-4bf5-b43a-ad99e9785595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:04.659619 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8521760-7510-4bf5-b43a-ad99e9785595 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:04.659895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8521760-7510-4bf5-b43a-ad99e9785595 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:04.660163 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8521760-7510-4bf5-b43a-ad99e9785595 None admin] PATH_INFO: `/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:04.661447 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8521760-7510-4bf5-b43a-ad99e9785595 None admin] RBAC: Authorizing `identity:get_project(project_id=45939c9fb3c94fad9f38a7a8fecd6af2)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:04.662909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8521760-7510-4bf5-b43a-ad99e9785595 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:04.664608 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 604/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:37:04 2022] GET /identity/v3/projects/45939c9fb3c94fad9f38a7a8fecd6af2 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:04.670891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:37:04.789634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:04.794135 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578173-f95d-49ef-9c68-4629d749ed6f None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:37:04.833789 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:37:04.834442 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:37:04.834720 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:37:04.835025 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:37:04.835853 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=45939c9fb3c94fad9f38a7a8fecd6af2, user.id=d8e39ea28fd841399d0569d456d3cbd6) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:37:04.836167 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:37:04.837624 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ca8e6-8df5-4c6c-a2e7-97781e974115 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:37:04.843736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 596/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:37:04 2022] GET /identity/v3/role_assignments?user.id=d8e39ea28fd841399d0569d456d3cbd6&scope.project.id=45939c9fb3c94fad9f38a7a8fecd6af2&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:37:14.652975 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=96 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:37:14.840994 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=18 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:03.065892 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92303-9097-4972-909b-c12405455ec5 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:03.066495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92303-9097-4972-909b-c12405455ec5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:03.066495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92303-9097-4972-909b-c12405455ec5 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:03.068263 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 605/1201] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:03.074424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:03.074957 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:03.074957 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:03.093536 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:03.094775 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:03.173335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 597/1202] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:03.191347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92303-9097-4972-909b-c12405455ec5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:03.311299 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92303-9097-4972-909b-c12405455ec5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:03.352974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:03.354351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:03.354984 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:03.355506 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:03.368234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:03.370215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:03.391479 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:03.392511 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:03.393478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:03.393478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:03.393478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:03.393478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:03.393478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 606/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:03.400091 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:03.509223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbe109-fe3a-475d-ad2b-c1a126f8e926 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:03.541271 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:03.548927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:03.549639 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:03.549922 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:03.550194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:03.550917 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:03.551223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:03.552667 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a201bd-7d2a-43a0-9906-357ace1f6e82 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:03.564697 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 598/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:03.570360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:03.634944 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:03.683496 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cead5f-3ffe-4f7b-8552-1ea9cacce300 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:03.716854 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd808eb-a29d-4776-954b-1652041b66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:03.717698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd808eb-a29d-4776-954b-1652041b66fb None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:03.717982 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd808eb-a29d-4776-954b-1652041b66fb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:03.718256 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd808eb-a29d-4776-954b-1652041b66fb None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:03.718967 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd808eb-a29d-4776-954b-1652041b66fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:03.721127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd808eb-a29d-4776-954b-1652041b66fb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:03.747466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 607/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:38:03 2022] POST /identity/v3/users => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:38:06.133630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a6a8-3ec8-40d3-b20a-27ae98d02ded None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.134200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a6a8-3ec8-40d3-b20a-27ae98d02ded None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.134200 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a6a8-3ec8-40d3-b20a-27ae98d02ded None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.135580 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 599/1206] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:06.141176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.141693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.141796 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.161617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:06.163032 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:06.243704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 608/1207] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:06.269274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357a6a8-3ec8-40d3-b20a-27ae98d02ded None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.387591 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357a6a8-3ec8-40d3-b20a-27ae98d02ded None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.428218 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.429031 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.429357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.429633 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.441618 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:06.449759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:06.450746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:06.451643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 600/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:06.457368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.504853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.576130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c716-71d0-4b84-b01c-cc729de4ca05 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.617972 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.618690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.618993 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.619313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.620103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:06.620416 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.621941 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.630897 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 609/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:06.637407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.692360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.745999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922899-8958-4438-847f-9d1fd6fe797c None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:06.794122 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:06.794989 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:06.795327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:06.795616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:06.807177 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:06.808650 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:06.827908 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:06.828903 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:06.830215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:06.830215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:06.830215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:06.830215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:06.830215 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 601/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:06.835546 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:06.958968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:06.963199 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf668a8-052e-477f-8765-f51d29e48fb9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.002025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.002701 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.002991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.003273 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.003964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:07.004267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.005811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.022392 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 610/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:07.024026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.146990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366e97a-4f11-4080-84af-fc5c66a69576 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.149557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.187416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.188214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.188492 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.188792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.197323 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.198842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:07.214263 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:07.215382 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:07.216456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 602/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:07.221341 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.342989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c790a0-68b1-4e8e-a80d-de7b72b7ceab None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.384168 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.385046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.385426 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.385649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.386478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:07.386780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.388239 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.412046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 611/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:07.413432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.535214 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20e04-9026-4a4e-b243-50e1e4eb45cc None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.580065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.580790 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.581105 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.581375 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.618113 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.671229 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.672861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.785360 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 603/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:38:07.787148 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:07.807138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:07.895642 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec2a42-afee-4aa3-ab99-9ebe00281b61 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:07.933761 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:07.934647 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:07.934999 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:07.935287 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:07.946917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:07.948501 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:07.968332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:07.969653 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:07.970571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:07.970571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:07.970571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:07.970571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:07.970571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 612/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:07.975587 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:08.047565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:08.092548 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe1211-8deb-43e8-96a9-268655566297 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:08.132403 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:08.133150 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:08.133626 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:08.133861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:08.134643 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:08.135005 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:08.136541 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:08.149856 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 604/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:08.155388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:08.238646 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:08.274660 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755a894-ad4d-45ef-9cd5-3b6a21a33971 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:08.317591 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:08.318374 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:08.318701 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:08.318980 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:08.329464 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:08.331372 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:08.347739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:08.348880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:08.349994 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:08.355946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 613/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:08.356086 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:08.478422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deb6a7-5c36-44f5-9c7e-8e9f568d6bf4 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:08.508276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:08.535705 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:08.536482 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:08.536804 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:08.537124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:08.538108 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:08.538486 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:08.540069 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201bba5-3c27-43c4-b2a4-b5a8a4a555e2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:08.556803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 605/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:08.563026 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:08.692455 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f0db-5b88-4762-80ed-d64723baefb1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:08.725782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:08.733159 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:08.734005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:08.734300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:08.734617 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:08.735635 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=2f5189fd89c0457da1f325be6bd6ff63) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:08.735937 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:08.737613 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df133-74c4-4d43-8613-a14560dcae28 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:08.744592 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 614/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:38:08 2022] GET /identity/v3/role_assignments?user.id=2f5189fd89c0457da1f325be6bd6ff63&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.263158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb097939-4f73-4619-8e7b-d03da0e8fad8 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.263779 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb097939-4f73-4619-8e7b-d03da0e8fad8 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.263779 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb097939-4f73-4619-8e7b-d03da0e8fad8 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.265095 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 606/1220] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:11.271176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.271674 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.271797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.290751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:11.291958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:11.369057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 615/1221] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:11.396100 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb097939-4f73-4619-8e7b-d03da0e8fad8 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.524061 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb097939-4f73-4619-8e7b-d03da0e8fad8 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.564963 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.565889 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.566194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.566510 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.578635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:11.586544 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:11.587579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:11.588509 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 607/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:11.594554 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.713405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f9ee-d192-49c4-a771-68eec79835c4 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.739571 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.752625 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.753339 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.753620 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.753890 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.754641 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:11.754942 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.756356 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.766264 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 616/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:11.772553 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:11.821621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:11.895838 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1ce9c-f657-4d62-8a31-676e6f77fc39 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:11.933270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:11.934175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:11.934463 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:11.934742 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:11.947345 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:11.948828 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:11.966306 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:11.967458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:11.968565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:11.968565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:11.968565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:11.968565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:11.968565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 608/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:11.974531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.089216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24e364-ac2c-4cca-99ac-74d7cf07d5a7 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.127746 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.128468 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.128792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.129095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.130002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.131250 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.131916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.144879 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 617/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.152021 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.256507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.264228 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72506b14-f3a2-491c-bb9b-642bb3a34fbd None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.298606 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.299426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.299707 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.299988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.309798 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.311366 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:12.327209 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:12.329655 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:12.333806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 609/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:12.335001 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.387477 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.444206 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579ef6c-8eea-488f-92d6-97292f61a1ac None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.485212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.486026 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.486322 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.486650 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.487578 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.487904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.490127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.509513 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 618/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.515430 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.630695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3baf-e8a9-4a62-aff8-1a636ce24fb2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.667709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.668456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.668745 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.669052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.681030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.682589 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:12.701346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:12.702751 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:12.703607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:12.703607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:12.703607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:12.703607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:12.703607 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 610/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:12.708894 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.822087 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d503eb-d2e1-4ef4-bcc3-96ae0bc7d7e9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:12.824865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:12.861765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:12.862597 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:12.862991 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:12.863267 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:12.864138 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:12.864473 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:12.866115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:12.878476 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 619/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:12.884493 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:12.895566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.003235 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06df4-31d6-44d0-98b8-ce5210914197 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.040020 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.040990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.041334 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.041639 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.049505 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.050981 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:13.066743 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:13.067927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:13.069076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 611/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:13.074595 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.185002 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35d3ec-9f27-4ad2-9377-756b32040adb None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.224985 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.225754 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.226117 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.226368 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.227283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:13.227743 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.229498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7dd68-8c84-4f2d-b9c2-bb829b4b7de1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.243698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 620/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:13.250316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:13.326576 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:13.364673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cdd12-2e6c-4841-9b25-a816e40c9d5a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:13.402531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8dfc-415e-4a74-9d56-818b016e061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:13.403444 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8dfc-415e-4a74-9d56-818b016e061f None admin] REQUEST_METHOD: `PUT` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:13.403702 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8dfc-415e-4a74-9d56-818b016e061f None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:13.405685 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8dfc-415e-4a74-9d56-818b016e061f None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/2f5189fd89c0457da1f325be6bd6ff63/roles/7656cbdaf70f4f0ba95fc59f85bb04a3` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:13.420781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a8dfc-415e-4a74-9d56-818b016e061f None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=2f5189fd89c0457da1f325be6bd6ff63, role_id=7656cbdaf70f4f0ba95fc59f85bb04a3)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:13.422276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a8dfc-415e-4a74-9d56-818b016e061f None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:13.436335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 612/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:38:13 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/2f5189fd89c0457da1f325be6bd6ff63/roles/7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:38:15.811853 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d92655-25e0-4b6f-b6d3-5f8f3c748812 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.812413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d92655-25e0-4b6f-b6d3-5f8f3c748812 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.812413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d92655-25e0-4b6f-b6d3-5f8f3c748812 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.813805 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 621/1233] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:15.819531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:15.820083 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:15.820194 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:15.837036 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:15.838241 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:15.908943 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 613/1234] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:15.933413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d92655-25e0-4b6f-b6d3-5f8f3c748812 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:15.960242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.048300 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d92655-25e0-4b6f-b6d3-5f8f3c748812 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.084064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.084825 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.085088 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.085390 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.095883 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:16.103778 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:16.104828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:16.105759 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 622/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:38:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:16.111560 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.166120 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.233039 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119147cb-a741-4c33-924c-a8147fb1cbf9 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.273918 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.274752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.275163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.275393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.276205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] RBAC: Adding query filter params (name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:16.276531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.277975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.287596 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 614/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:16.293670 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.386401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.407362 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba67df-428c-42d1-aac2-9afd3da35371 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.444659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.445580 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.445863 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.446125 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.464669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.466242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:16.485658 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:16.486659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:16.487527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:16.487527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:16.487527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:16.487527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:16.487527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 623/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:16.493362 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.611258 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd320fd-bc51-41db-b9b5-d678299f2dfe None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.621023 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.651843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.652450 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.652727 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.653000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.653773 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:16.654070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.655459 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.667424 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 615/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:16.674041 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.792801 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3bbea-14c9-4cb6-b9fb-be7913adaa53 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:16.828528 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:16.829523 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:16.829835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:16.830158 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:16.833383 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:16.843318 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:16.846331 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:16.869092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:16.870271 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:16.871648 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 624/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:16.876516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:16.986990 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f38ad8-f75c-450f-8c0d-9458ec57006e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.024156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.024851 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.025133 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.025439 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.026289 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:17.026591 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.028029 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.047862 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 616/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:38:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:17.053952 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.168212 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae8582-036a-460c-b915-a59591c1d0d3 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.205550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.206238 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.206521 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.206900 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.275460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.282541 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.284154 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.396101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 625/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:38:17.397473 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.428842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.519372 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f483a-7974-408d-a1b0-4562819d436d None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.558162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.558900 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.559171 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.559437 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.570924 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.571335 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:17.590207 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:17.591168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:17.592785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:17.592785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:17.592785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:17.592785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:17.592785 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 617/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:17.597417 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.702412 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.710286 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049486a8-a832-44bf-a69c-2d79964bccbe None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.750983 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.751656 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.751949 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.752201 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.752920 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:17.753219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.754631 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.766612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 626/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:17.773357 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:17.864452 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:17.891531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cecebf-c246-466e-b1e3-3cf976c7fc44 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:17.929862 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:17.930679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:17.930952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:17.931249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:17.938097 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:17.939551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:17.953159 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:17.954247 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:17.955178 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 618/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:17.960787 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.072013 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8bf2-acd3-4ae2-a996-87d93ed4164e None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.106698 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.107347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.107636 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.107947 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.108793 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.109127 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.110752 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8167206-85e1-4895-8ed4-7e0972bfd3e7 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.123492 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 627/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:18.130219 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:18.246081 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c086b-5136-42f9-93a0-2bcf407a1088 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:18.286089 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:18.287122 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:18.287454 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:18.287866 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:18.289015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] RBAC: Adding query filter params (role.id=7656cbdaf70f4f0ba95fc59f85bb04a3, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=2f5189fd89c0457da1f325be6bd6ff63) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:18.289397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:18.291320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221251ae-433f-48ac-bcef-2fcbbfcc157b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:18.309407 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 619/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:38:18 2022] GET /identity/v3/role_assignments?user.id=2f5189fd89c0457da1f325be6bd6ff63&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=7656cbdaf70f4f0ba95fc59f85bb04a3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:20.803762 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ddb9-c301-4a09-a854-34f84add4bd8 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.804295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ddb9-c301-4a09-a854-34f84add4bd8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.804295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ddb9-c301-4a09-a854-34f84add4bd8 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.805964 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 628/1247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:20.812328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:20.812887 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:20.813163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:20.818694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:20.831964 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:20.833184 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:20.908525 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 620/1248] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:20.934208 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835ddb9-c301-4a09-a854-34f84add4bd8 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:20.955728 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.058450 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835ddb9-c301-4a09-a854-34f84add4bd8 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.098989 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.099844 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.100176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.100448 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.111484 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:21.119399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:21.120530 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:21.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 629/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:38:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:21.126865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.243433 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a32ce5-1e67-42e5-b677-f493b11685ee None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.280539 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.286520 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.287256 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.287537 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.287808 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.288703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:21.289006 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.290622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.300646 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 621/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:21.306794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.394734 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.422007 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5b2af-a15b-4780-b29d-098fa9d49c7d None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.461421 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.462366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.462650 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.462952 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.474856 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.476339 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:21.495951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:21.497057 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:21.498097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:21.498097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:21.498097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:21.498097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:21.498097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 630/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:21.503703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.612806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c482-ad57-421b-a8b3-dadd498dfa20 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.652419 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.653024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.653328 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.653652 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.654390 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:21.654703 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.656086 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.669058 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 622/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:21.675020 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.786577 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfb97-7a06-4120-a648-5a14215a5d05 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:21.816389 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.825828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:21.826682 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:21.826992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:21.827280 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] PATH_INFO: `/v3/users/placement` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:21.836131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:21.837806 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:21.853101 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:21.855097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:21.856694 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 631/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:21.861471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:21.924325 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:21.971679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55441212-83f0-400b-b2b1-0294e30f51cb None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.008015 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.008699 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.008978 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.009272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.010124 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:22.010426 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.011950 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.030166 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 623/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:38:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:22.036460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.145913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b886aef-f89f-4dca-8ca3-f80ec31f309f None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.188253 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.188828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.189105 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.189464 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.253812 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.267235 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.269000 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.391909 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 632/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:38:22.398980 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.516483 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e994df-0f95-46f5-9195-3c44e2ed7ee2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.531134 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.554693 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.555570 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.555852 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.556257 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.568860 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.570363 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:22.590180 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:22.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:22.596759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:22.596759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:22.596759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:22.596759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:22.596759 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 624/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:22.597348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.692559 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:22.705382 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478975-213f-41cb-a53f-3665a79ddd61 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.743661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.744531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.744848 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.745135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.746137 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:22.746516 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.748195 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.762145 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 633/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:22.768061 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:22.876438 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44f231-c0c3-4f71-8c14-00c811a4c76e None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:22.914748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:22.915926 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:22.916274 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:22.916531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:22.923793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:22.925270 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:22.941383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:22.942573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:22.943593 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 625/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:22.949357 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.059348 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efc0ec-c712-42c5-b4c2-f5fce53e19d9 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.102149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.102917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.103209 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.103529 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] PATH_INFO: `/v3/projects` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.104366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:23.104684 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.106343 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866751e-3390-4b85-b55a-2c76aaefdb72 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.120336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 634/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:23.126738 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:23.157678 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:23.240862 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f51b5-22b0-490e-b364-ecf7d633ea54 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:23.278638 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:23.279305 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:23.279598 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:23.279853 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:23.280682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=2f5189fd89c0457da1f325be6bd6ff63) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:23.280992 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:23.282462 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550b590-b526-4eeb-80fc-dd345015fbaa None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:23.288924 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 626/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:38:23 2022] GET /identity/v3/role_assignments?user.id=2f5189fd89c0457da1f325be6bd6ff63&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:25.559858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650ff9-a56c-4754-86b3-4f055cec868b None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.560395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650ff9-a56c-4754-86b3-4f055cec868b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.560395 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650ff9-a56c-4754-86b3-4f055cec868b None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.561882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 635/1261] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:25.568356 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.569070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.569221 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.598332 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:25.599861 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:25.671766 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 627/1262] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:25.702804 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db650ff9-a56c-4754-86b3-4f055cec868b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.715405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.811413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db650ff9-a56c-4754-86b3-4f055cec868b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:25.846588 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:25.847375 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:25.847651 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:25.847917 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:25.858629 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:25.866242 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:25.867332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:25.868284 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 636/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:25.873469 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:25.892312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:25.983232 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1ed6-10c4-4f04-b300-7d5b8ce89512 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.019613 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.020497 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.021017 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.021566 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] PATH_INFO: `/v3/roles` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.023316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.023849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.026341 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.035266 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 628/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:38:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:26.041683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.121152 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.155810 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19ee5-08cd-4492-98fe-80b2a2b5d89c None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.190603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.191401 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.191679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.191946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.204213 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.205893 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:26.225775 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.226816 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.227704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:26.227704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:26.227704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:26.227704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:26.227704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 637/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.233900 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.321834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.352656 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22024e74-e5dc-4f08-9127-d839f8cf46c5 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.389354 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.389969 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.390250 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.390526 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.391252 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.391563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.392982 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.405665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 629/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.412566 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.522104 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe10347-8d33-4353-8fd7-eeb68feec5fa None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.542098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.557375 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.558189 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.558471 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.558739 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.566925 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.568494 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:26.582589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.583612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:26.584691 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 638/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.589877 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.702971 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2b72e-1c39-4b60-ad5e-800508b54c5b None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.739835 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.740528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.740806 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.741079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] PATH_INFO: `/v3/users` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.741945 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:26.742249 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.743679 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.752726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:26.767292 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 630/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:26.774336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:26.888120 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3929-085f-46b9-8db1-71b7e0f3e213 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:26.923174 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:26.923958 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:26.924232 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:26.924499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:26.935350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:26.936808 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:26.954089 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:26.955065 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:26.956090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:26.956090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:26.956090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:26.956090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:26.956090 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 639/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:26.961195 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.070432 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257a63-12e2-44e7-860f-dc7f1eb8d482 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.106878 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.107536 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.107815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.108079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.108791 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:27.109085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.110476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.121839 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 631/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:27.127616 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.164231 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.234413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab0f83-6201-4f7d-a2c4-122ba6d2baa1 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.272005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.273339 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.273621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.273895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.282262 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.284495 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:27.299459 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:27.300460 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:27.301531 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 640/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:27.308298 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.370567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.414373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53311d3-f8b2-4494-8603-ce87af0e4cae None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.453175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.453871 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.454153 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.454416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.455212 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:27.455517 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.456967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2def-732c-4a93-9092-053b3b7475c5 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.470244 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 632/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:27.476606 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:27.576581 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:27.587063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b158b3-1b04-4735-865f-15327b1f9637 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:27.625852 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38e41f-1142-4486-b8b6-221fefeeaf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:27.626761 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38e41f-1142-4486-b8b6-221fefeeaf7c None admin] REQUEST_METHOD: `PUT` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:27.627034 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38e41f-1142-4486-b8b6-221fefeeaf7c None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:27.627307 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38e41f-1142-4486-b8b6-221fefeeaf7c None admin] PATH_INFO: `/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/2f5189fd89c0457da1f325be6bd6ff63/roles/788439e070434cb1abe99a58ff6bfa36` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:27.629578 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38e41f-1142-4486-b8b6-221fefeeaf7c None admin] RBAC: Authorizing `identity:create_grant(project_id=1e03681ba5554dc2ab23b1df5b3b4762, user_id=2f5189fd89c0457da1f325be6bd6ff63, role_id=788439e070434cb1abe99a58ff6bfa36)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:27.631066 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38e41f-1142-4486-b8b6-221fefeeaf7c None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:27.647115 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 641/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:38:27 2022] PUT /identity/v3/projects/1e03681ba5554dc2ab23b1df5b3b4762/users/2f5189fd89c0457da1f325be6bd6ff63/roles/788439e070434cb1abe99a58ff6bfa36 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:38:29.843988 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8b503-3186-49d4-88bd-eee66ed604db None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.844528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8b503-3186-49d4-88bd-eee66ed604db None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.844528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8b503-3186-49d4-88bd-eee66ed604db None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.845881 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 633/1274] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:29.852432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:29.852885 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:29.852987 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:29.875465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:29.876677 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:29.958413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 642/1275] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:29.982334 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8b503-3186-49d4-88bd-eee66ed604db None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.099952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8b503-3186-49d4-88bd-eee66ed604db None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.118290 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.135823 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.136647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.136929 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.137202 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.147891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:38:30.155834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:30.156901 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:30.162373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 634/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:30.164740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.255982 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.275748 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbcfcf-519f-4ec4-a578-4889e7eae27d None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.324180 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.324847 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.325131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.325436 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] PATH_INFO: `/v3/roles` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.326184 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:30.326487 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.327878 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.336478 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 643/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.342867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.450665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449cc63-71d7-4ab9-9913-2ed6ccd68404 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.486464 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.487264 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.487542 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.487815 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.499276 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.500746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:30.518310 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:30.519287 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:30.520316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:30.520316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:30.520316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:30.520316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:30.520316 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 635/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:30.528599 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.640842 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8e17-baf2-4c0c-8ccb-233f67bc0360 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.676045 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.676722 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.677023 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.677333 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] PATH_INFO: `/v3/domains` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.678063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:30.678360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.679751 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.691572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 644/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:30.698131 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.734261 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.806429 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdba44-e0de-4628-a476-858530f173af None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:30.842400 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:30.843196 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:30.843475 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:30.843748 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:30.851557 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:30.853010 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:38:30.867724 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:30.868780 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:30.870471 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 636/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:30.878043 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:30.882527 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:30.983797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d83bbf-25f0-4915-9c4d-330f12699378 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.019486 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.020092 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.020391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.020661 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] PATH_INFO: `/v3/users` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.021525 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:31.021835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.023645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.040787 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 645/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:38:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:31.047161 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.165810 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.178334 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456de63-0410-4ce2-89e4-4f97a177c2f6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.215456 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.216142 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.216521 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.216864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.290972 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.292369 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.387000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.398715 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 637/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:38:31.404881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.478313 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.520615 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa302-1451-42f0-a750-a3b4643090e5 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.560704 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.561481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.561765 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.562035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.572740 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.574234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:38:31.592347 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:31.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:31.594870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:38:31.594870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:38:31.594870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:38:31.594870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:31.594870 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 646/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:31.601374 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.703622 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2304d37-6c1d-44a5-9589-56afe5108326 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.740867 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.741669 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.741998 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.742329 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] PATH_INFO: `/v3/domains` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.743210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:31.743523 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.745067 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.757208 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 638/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:38:31.765584 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:31.877669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a673-cab7-43dd-a49d-e54054dfe2c6 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:31.891711 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:31.921861 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:31.922665 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:31.922948 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:31.923219 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:31.930335 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:31.931839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:38:31.945758 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application with Lock( May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:31.946688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: ERROR keystone.server.flask.application May 19 14:38:31.947706 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 647/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:38:31.953671 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.015792 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:32.067143 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8a86b-17b7-4b11-8614-4c07094a37d6 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.105842 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.106507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.106787 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.107062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] PATH_INFO: `/v3/projects` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.107849 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.108154 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.109629 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e9ce3-14f1-449b-b2c9-fce51690151e None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.122030 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 639/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:32.132782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:32.250688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070334f-627f-47ee-ad43-b12cc403bd59 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:32.300585 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:32.301811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:32.302266 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:32.302583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:32.303829 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] RBAC: Adding query filter params (role.id=788439e070434cb1abe99a58ff6bfa36, scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762, user.id=2f5189fd89c0457da1f325be6bd6ff63) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:32.304255 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:32.306599 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd33aa9-cf42-4d9e-84d3-3ec0c3d818ac None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:32.317093 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 648/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:38:32 2022] GET /identity/v3/role_assignments?user.id=2f5189fd89c0457da1f325be6bd6ff63&scope.project.id=1e03681ba5554dc2ab23b1df5b3b4762&role.id=788439e070434cb1abe99a58ff6bfa36 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:34.532421 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fdf48-6b4b-486f-915d-1f631ba695ec None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:34.532967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fdf48-6b4b-486f-915d-1f631ba695ec None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:34.532967 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fdf48-6b4b-486f-915d-1f631ba695ec None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:34.534425 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 640/1288] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:34.540645 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:34.541283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:34.541283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:34.567859 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:34.569063 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:34.603587 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:34.647109 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 649/1289] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:34.670322 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fdf48-6b4b-486f-915d-1f631ba695ec None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:34.796621 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fdf48-6b4b-486f-915d-1f631ba695ec None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:34.813755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:34.838824 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:34.839625 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:34.839912 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:34.840181 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:34.840834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:34.842251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:34.854752 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:34.856079 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:34.856927 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 641/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:38:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:34.864828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:34.971600 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c0fc4-f639-4ae3-b879-17363c1ecc71 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:35.007293 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:35.007955 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.008234 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.008498 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.009204 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:35.009543 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:35.010929 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8706cba-6bc5-4737-92a3-1536493374a9 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:35.015951 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 650/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:38:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:35.022400 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:35.137935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f429ee-c12a-491d-b085-d911d84c5cf2 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:35.176817 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:35.177530 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:35.177875 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:35.178160 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] PATH_INFO: `/v3/services` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:35.178865 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] RBAC: Adding query filter params (type=placement) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:35.179175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:35.180565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688e7c6-e0dd-4910-a49a-24ec706f9649 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:35.185271 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 642/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:38:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:37.381786 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37aa39-d714-4a1f-97f8-0b06cdb678c8 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.382355 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37aa39-d714-4a1f-97f8-0b06cdb678c8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.382355 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37aa39-d714-4a1f-97f8-0b06cdb678c8 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.384593 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 651/1293] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:37.394515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236d2b6-9a28-4001-be27-d76b1e8a57af None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.394918 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236d2b6-9a28-4001-be27-d76b1e8a57af None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.395140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236d2b6-9a28-4001-be27-d76b1e8a57af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.415853 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-f236d2b6-9a28-4001-be27-d76b1e8a57af None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:37.417076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-f236d2b6-9a28-4001-be27-d76b1e8a57af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:37.449673 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.493567 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 643/1294] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:37.527517 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37aa39-d714-4a1f-97f8-0b06cdb678c8 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:37.563744 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:37.638139 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37aa39-d714-4a1f-97f8-0b06cdb678c8 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:37.675749 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea6fa-ebda-4107-a9dc-009798816b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:37.676414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea6fa-ebda-4107-a9dc-009798816b25 None admin] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:37.676695 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea6fa-ebda-4107-a9dc-009798816b25 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:37.676969 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea6fa-ebda-4107-a9dc-009798816b25 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:37.677774 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea6fa-ebda-4107-a9dc-009798816b25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:37.679208 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea6fa-ebda-4107-a9dc-009798816b25 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:37.694520 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 652/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:38:37 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:38:39.898060 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6b733-638f-47f0-84da-ea7163bdc92a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.898891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6b733-638f-47f0-84da-ea7163bdc92a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.898891 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6b733-638f-47f0-84da-ea7163bdc92a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.901053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 644/1296] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:39.909256 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:39.909782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:39.909889 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:39.927576 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:39.928828 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:40.007835 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 653/1297] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:38:40.031170 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6b733-638f-47f0-84da-ea7163bdc92a None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.156126 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6b733-638f-47f0-84da-ea7163bdc92a None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.192355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.193158 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.193472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.193746 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.194405 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.195781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:40.207944 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:40.209024 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:40.209911 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 645/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:40.217536 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.237572 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.322466 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d90364-fa42-483e-ae95-0864eff5597b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.358657 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.359317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.359542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.359850 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.360550 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:40.360849 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.362258 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.367699 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 654/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:40.377448 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.417137 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.492813 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d40cc-931e-4b1b-a089-2a0280f8e731 None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.533846 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.534507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.534781 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.535048 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.535829 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] RBAC: Adding query filter params (interface=public, service_id=28d8f6c6a8734cd69bcbd27fca6d2e7e, region_id=RegionOne) {{(pid=150372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:40.536140 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.537651 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206dea9-dec0-43ff-81f4-6016eceab12b None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.542663 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 646/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/endpoints?service_id=28d8f6c6a8734cd69bcbd27fca6d2e7e&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:40.551881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:40.652157 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:40.672099 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5ea17-aa12-4a20-ad12-219b4397e039 None admin] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:40.706316 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c611a-70a5-460c-b9c1-4053d35b599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:40.706988 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c611a-70a5-460c-b9c1-4053d35b599e None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:40.707268 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c611a-70a5-460c-b9c1-4053d35b599e None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:40.707539 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c611a-70a5-460c-b9c1-4053d35b599e None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:40.708221 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c611a-70a5-460c-b9c1-4053d35b599e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:40.709683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c611a-70a5-460c-b9c1-4053d35b599e None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:40.716423 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 655/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:38:40 2022] GET /identity/v3/services => generated 2191 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:38:43.004175 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93149d8-0323-4e16-81c1-553c9f43fd77 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.004734 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93149d8-0323-4e16-81c1-553c9f43fd77 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.004734 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93149d8-0323-4e16-81c1-553c9f43fd77 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.006223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 647/1302] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:38:43.012683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.013486 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.013649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.040482 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:38:43.041737 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:38:43.112442 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 656/1303] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:38:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:38:43.142923 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93149d8-0323-4e16-81c1-553c9f43fd77 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.247476 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.259812 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93149d8-0323-4e16-81c1-553c9f43fd77 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.300586 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.301409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.301682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.301952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] PATH_INFO: `/v3/services/placement` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.302635 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.304000 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:38:43.316314 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:38:43.317551 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application with Lock( May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self._enter() May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application return self.creator() May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application created_value = creator( May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: ERROR keystone.server.flask.application May 19 14:38:43.318952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 648/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:38:43.327064 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.415907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:43.444380 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56454f-62b5-4b05-9b4c-0baa21c51dc3 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.487130 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.487794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.488067 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.488334 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] PATH_INFO: `/v3/services` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.489035 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:38:43.489366 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.490845 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa630ebf-25ef-40e9-9c77-028bb518c537 None admin] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.495974 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 657/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:38:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:38:43.502326 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:38:43.606272 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed0b66-b993-4522-a968-e1f4bde1418a None admin] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:38:43.642682 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9be83-8183-4c94-bd9a-24d9d1a7b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ad0c94b4942efa0beedd41caa0934', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:38:43.643327 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9be83-8183-4c94-bd9a-24d9d1a7b732 None admin] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:38:43.643599 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9be83-8183-4c94-bd9a-24d9d1a7b732 None admin] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:38:43.643864 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9be83-8183-4c94-bd9a-24d9d1a7b732 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:38:43.644602 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9be83-8183-4c94-bd9a-24d9d1a7b732 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:38:43.646019 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9be83-8183-4c94-bd9a-24d9d1a7b732 None admin] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:38:43.667320 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 649/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:38:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:38:53.492921 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=30 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:38:53.663466 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:49.074131 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd5c63-5625-43ae-b87d-2094f9702c84 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.074696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd5c63-5625-43ae-b87d-2094f9702c84 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.074696 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd5c63-5625-43ae-b87d-2094f9702c84 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.075992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 658/1307] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:49.081519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22360216-5bc7-4deb-8423-60d0ccf50c29 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.081970 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22360216-5bc7-4deb-8423-60d0ccf50c29 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.082053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22360216-5bc7-4deb-8423-60d0ccf50c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.210741 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-22360216-5bc7-4deb-8423-60d0ccf50c29 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:49.212056 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-22360216-5bc7-4deb-8423-60d0ccf50c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:49.339065 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 650/1308] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:39:49.410327 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2d124-59a2-4e2a-ad36-861e75773cfc None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.410669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2d124-59a2-4e2a-ad36-861e75773cfc None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.410887 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2d124-59a2-4e2a-ad36-861e75773cfc None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.412329 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 659/1309] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 14:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:49.429965 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc07f7-e6d3-4e14-9304-8b1e0b13b8b4 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:49.430302 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc07f7-e6d3-4e14-9304-8b1e0b13b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:49.430522 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc07f7-e6d3-4e14-9304-8b1e0b13b8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:49.627561 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-6dbc07f7-e6d3-4e14-9304-8b1e0b13b8b4 None None] MFA Rules not processed for user `bafd4f61582941a88f68e01deb85d4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:49.628946 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-6dbc07f7-e6d3-4e14-9304-8b1e0b13b8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:49.656514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:49.727664 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 651/1310] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:39:49.736992 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2d124-59a2-4e2a-ad36-861e75773cfc None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:49.739046 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-f3c2d124-59a2-4e2a-ad36-861e75773cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:49.953270 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2d124-59a2-4e2a-ad36-861e75773cfc None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:50.072841 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:50.077669 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:50.078655 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:50.079215 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:50.081379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:50.146874 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:50.214098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:50.217042 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06a142-6104-4c65-86c3-6e44ea513e56 service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:50.242467 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 660/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:39:52.801708 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0cb86-a441-4b33-b4c0-6b1da8dab031 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.802279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0cb86-a441-4b33-b4c0-6b1da8dab031 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.802279 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0cb86-a441-4b33-b4c0-6b1da8dab031 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.803869 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 652/1312] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:52.809433 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8437b0-b027-4977-ae3e-6fc411b852a4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:52.809881 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8437b0-b027-4977-ae3e-6fc411b852a4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:52.810098 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8437b0-b027-4977-ae3e-6fc411b852a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:52.887217 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4c8437b0-b027-4977-ae3e-6fc411b852a4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:52.888440 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4c8437b0-b027-4977-ae3e-6fc411b852a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:53.009388 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 661/1313] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:39:53.042578 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0cb86-a441-4b33-b4c0-6b1da8dab031 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:53.044151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-c6a0cb86-a441-4b33-b4c0-6b1da8dab031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:53.162284 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0cb86-a441-4b33-b4c0-6b1da8dab031 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:53.240157 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:53.244216 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:53.244897 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:53.245187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:53.245513 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:53.247975 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:53.378324 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:53.379834 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be72c4-d3b9-4057-b8db-8fb726f0ac3c service neutron] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:53.402119 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 653/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:39:55.973926 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0421-5fc2-477d-875b-9b6b2ece7933 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:55.974573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0421-5fc2-477d-875b-9b6b2ece7933 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:55.974573 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0421-5fc2-477d-875b-9b6b2ece7933 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:55.976051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 662/1315] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:55.982338 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c05b2-965e-417a-a046-34ccba3ed1ef None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:55.982862 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c05b2-965e-417a-a046-34ccba3ed1ef None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:55.983033 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c05b2-965e-417a-a046-34ccba3ed1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:56.056916 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-c17c05b2-965e-417a-a046-34ccba3ed1ef None None] MFA Rules not processed for user `9992281c74bc46bab68fdf142522a8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:56.058213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-c17c05b2-965e-417a-a046-34ccba3ed1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:56.116554 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:56.165803 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 654/1316] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:39:56.205895 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb0421-5fc2-477d-875b-9b6b2ece7933 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:56.207545 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6efb0421-5fc2-477d-875b-9b6b2ece7933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:56.219140 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:56.319071 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb0421-5fc2-477d-875b-9b6b2ece7933 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:56.398551 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:56.399135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:56.399418 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:56.399690 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:56.401405 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:56.466782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:56.518077 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:56.519605 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f182b5-9664-4589-9f6e-609600976f5d service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:56.543718 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 663/1317] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:39:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:39:59.329213 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263eb4a-8f0b-4f98-915c-631247ba9ee3 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:59.329814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263eb4a-8f0b-4f98-915c-631247ba9ee3 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:59.329814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263eb4a-8f0b-4f98-915c-631247ba9ee3 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:59.331193 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 655/1318] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:39:59.337780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e0864-358a-4c29-9f63-87bc13d548c8 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:59.338503 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e0864-358a-4c29-9f63-87bc13d548c8 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:59.338683 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e0864-358a-4c29-9f63-87bc13d548c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:59.409729 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-4c8e0864-358a-4c29-9f63-87bc13d548c8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:39:59.410933 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-4c8e0864-358a-4c29-9f63-87bc13d548c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:59.528332 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 664/1319] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:39:59.567187 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263eb4a-8f0b-4f98-915c-631247ba9ee3 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:39:59.568831 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-2263eb4a-8f0b-4f98-915c-631247ba9ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:59.681639 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263eb4a-8f0b-4f98-915c-631247ba9ee3 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:39:59.756960 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:39:59.757634 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:39:59.757885 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:39:59.758149 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:39:59.759814 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:39:59.767396 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:39:59.874528 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:39:59.875999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60561891-7093-4a95-b68d-339b67fd6c2e service neutron] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:39:59.897843 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 656/1320] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:02.802965 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e088f-c7e7-4d29-9a62-fc518c28605d None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:02.803499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e088f-c7e7-4d29-9a62-fc518c28605d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:02.803499 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e088f-c7e7-4d29-9a62-fc518c28605d None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:02.804791 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 665/1321] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:02.810197 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8880a-e8d7-477b-87f5-f910ff8e08b9 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:02.810616 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8880a-e8d7-477b-87f5-f910ff8e08b9 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:02.810710 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8880a-e8d7-477b-87f5-f910ff8e08b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:02.877755 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-dbc8880a-e8d7-477b-87f5-f910ff8e08b9 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:02.878999 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-dbc8880a-e8d7-477b-87f5-f910ff8e08b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:02.939833 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:02.981597 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 657/1322] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:03.033324 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6400f-4014-467a-8f1f-c906ef1f037e None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:03.033771 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6400f-4014-467a-8f1f-c906ef1f037e None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:03.034051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6400f-4014-467a-8f1f-c906ef1f037e None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:03.035465 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 666/1323] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 14:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:03.042070 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb3201-fae8-444f-85e9-edccb7b3b053 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:03.042364 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb3201-fae8-444f-85e9-edccb7b3b053 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:03.042653 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb3201-fae8-444f-85e9-edccb7b3b053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:03.132078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-01eb3201-fae8-444f-85e9-edccb7b3b053 None None] MFA Rules not processed for user `bafd4f61582941a88f68e01deb85d4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:03.133299 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-01eb3201-fae8-444f-85e9-edccb7b3b053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:03.218173 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 658/1324] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:03.227888 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6400f-4014-467a-8f1f-c906ef1f037e None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:03.229451 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6ff6400f-4014-467a-8f1f-c906ef1f037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:03.248777 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:03.338971 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6400f-4014-467a-8f1f-c906ef1f037e None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:03.415097 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:03.415668 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:03.415952 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:03.416226 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:03.417907 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:03.490957 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:03.532745 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:03.534603 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751cede-3f67-458e-a30f-a5fe6532157f service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:03.552930 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 667/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:06.026668 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cbfbc-55b5-448a-931a-3db8de1dd820 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:06.027262 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cbfbc-55b5-448a-931a-3db8de1dd820 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:06.027262 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cbfbc-55b5-448a-931a-3db8de1dd820 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:06.028581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 659/1326] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:06.034225 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd8c0-bdad-4930-acbf-3dad1ae49bb4 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:06.034623 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd8c0-bdad-4930-acbf-3dad1ae49bb4 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:06.034797 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd8c0-bdad-4930-acbf-3dad1ae49bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:06.102668 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-646cd8c0-bdad-4930-acbf-3dad1ae49bb4 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:06.103902 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-646cd8c0-bdad-4930-acbf-3dad1ae49bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:06.206122 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 668/1327] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:06.242436 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cbfbc-55b5-448a-931a-3db8de1dd820 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:06.243995 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-897cbfbc-55b5-448a-931a-3db8de1dd820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:06.308085 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:06.353577 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cbfbc-55b5-448a-931a-3db8de1dd820 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:06.431006 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:06.431666 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:06.431952 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:06.432223 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:06.433904 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:06.549726 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:06.552499 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:06.554592 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891ec4b-bbb0-4cde-b112-b4246039a6de service neutron] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:06.574331 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 660/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:08.831312 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845160f6-68c4-4e4a-a9aa-35f96a43854a None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:08.831839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845160f6-68c4-4e4a-a9aa-35f96a43854a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:08.831839 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845160f6-68c4-4e4a-a9aa-35f96a43854a None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:08.833210 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 669/1329] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:08.839695 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c0559-0ef0-401e-ab4e-9b30423644ad None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:08.840355 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c0559-0ef0-401e-ab4e-9b30423644ad None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:08.840579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c0559-0ef0-401e-ab4e-9b30423644ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:08.918514 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-1a2c0559-0ef0-401e-ab4e-9b30423644ad None None] MFA Rules not processed for user `9992281c74bc46bab68fdf142522a8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:08.919735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-1a2c0559-0ef0-401e-ab4e-9b30423644ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:09.031251 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 661/1330] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:09.070227 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845160f6-68c4-4e4a-a9aa-35f96a43854a None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:09.071780 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-845160f6-68c4-4e4a-a9aa-35f96a43854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:09.112916 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:09.195953 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845160f6-68c4-4e4a-a9aa-35f96a43854a None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:09.276649 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:09.277301 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:09.277621 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:09.277890 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:09.279542 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:09.369070 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:09.412913 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:09.414411 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8916ed6-3d4e-4378-83b0-9ad4e482f820 service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:09.432821 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 670/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:12.043416 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124816ac-f5e1-4094-91c5-688e3120de20 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:12.044346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124816ac-f5e1-4094-91c5-688e3120de20 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:12.044346 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124816ac-f5e1-4094-91c5-688e3120de20 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:12.046686 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 662/1332] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:12.053507 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdbf0a-f999-40d1-8687-e211470e45e3 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:12.053939 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdbf0a-f999-40d1-8687-e211470e45e3 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:12.054102 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdbf0a-f999-40d1-8687-e211470e45e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:12.128173 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-affdbf0a-f999-40d1-8687-e211470e45e3 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:12.129413 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-affdbf0a-f999-40d1-8687-e211470e45e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:12.241351 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 671/1333] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:12.278936 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124816ac-f5e1-4094-91c5-688e3120de20 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:12.281422 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-124816ac-f5e1-4094-91c5-688e3120de20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:12.356095 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:12.395026 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124816ac-f5e1-4094-91c5-688e3120de20 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:12.478458 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:12.479112 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:12.479397 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:12.479675 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:12.481445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:12.594370 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:12.595819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e7819-e5bd-4720-b98f-b48a87a899d9 service neutron] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:12.606168 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:12.615507 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 663/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:14.850243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93934-5bdd-4fa7-9026-8053bf582b75 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:14.850811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93934-5bdd-4fa7-9026-8053bf582b75 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:14.850811 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93934-5bdd-4fa7-9026-8053bf582b75 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:14.852192 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 672/1335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:14.858811 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cc681-75b0-4d60-9c00-119dba50602b None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:14.859563 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cc681-75b0-4d60-9c00-119dba50602b None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:14.859741 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cc681-75b0-4d60-9c00-119dba50602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:14.934515 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-037cc681-75b0-4d60-9c00-119dba50602b None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:14.935733 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-037cc681-75b0-4d60-9c00-119dba50602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:15.039411 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 664/1336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:15.080426 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93934-5bdd-4fa7-9026-8053bf582b75 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:15.082995 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-5ad93934-5bdd-4fa7-9026-8053bf582b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:15.131721 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:15.203782 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93934-5bdd-4fa7-9026-8053bf582b75 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:15.284767 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:15.285481 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:15.285785 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:15.286051 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:15.287968 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:15.383541 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:15.418882 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:15.420360 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5b4d3-8f8c-44a7-ab55-336f5b9adfdb service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:15.441040 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 673/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:17.991237 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfa710-c6ea-4973-8c34-7fc2b1f02513 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:17.992163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfa710-c6ea-4973-8c34-7fc2b1f02513 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:17.992163 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfa710-c6ea-4973-8c34-7fc2b1f02513 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:17.994516 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 665/1338] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:18.001058 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66487c1f-9fdc-41e7-8e1e-f9b3b51e4c52 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:18.001865 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66487c1f-9fdc-41e7-8e1e-f9b3b51e4c52 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:18.002044 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66487c1f-9fdc-41e7-8e1e-f9b3b51e4c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:18.073432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-66487c1f-9fdc-41e7-8e1e-f9b3b51e4c52 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:18.074709 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-66487c1f-9fdc-41e7-8e1e-f9b3b51e4c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:18.178039 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 674/1339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:18.214694 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfa710-c6ea-4973-8c34-7fc2b1f02513 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:18.216478 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-0ebfa710-c6ea-4973-8c34-7fc2b1f02513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:18.297294 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:18.328630 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfa710-c6ea-4973-8c34-7fc2b1f02513 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:18.404935 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:18.405620 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:18.405896 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:18.406172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:18.407819 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:18.538250 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:18.539739 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529dc32-8240-4025-90aa-46979b38734c service neutron] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:18.558850 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 666/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:22.971688 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf9f59-42b5-411e-829f-4efbe802231a None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:22.972283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf9f59-42b5-411e-829f-4efbe802231a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:22.972283 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf9f59-42b5-411e-829f-4efbe802231a None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:22.973675 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 675/1341] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:22.979468 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579d6a3-0a28-4cc5-906d-9f3a90dfb251 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:22.979958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579d6a3-0a28-4cc5-906d-9f3a90dfb251 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:22.980067 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579d6a3-0a28-4cc5-906d-9f3a90dfb251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:22.986472 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=100 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:23.053663 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-0579d6a3-0a28-4cc5-906d-9f3a90dfb251 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:23.054805 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-0579d6a3-0a28-4cc5-906d-9f3a90dfb251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:23.162519 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 667/1342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:23.201867 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf9f59-42b5-411e-829f-4efbe802231a None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:23.205340 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-5ccf9f59-42b5-411e-829f-4efbe802231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:23.257517 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:23.305409 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf9f59-42b5-411e-829f-4efbe802231a None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:23.309230 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f305465-ba90-4432-850f-322de842ccda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:23.309904 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305465-ba90-4432-850f-322de842ccda service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:23.310177 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305465-ba90-4432-850f-322de842ccda service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:23.310444 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305465-ba90-4432-850f-322de842ccda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:23.319679 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305465-ba90-4432-850f-322de842ccda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:23.321117 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305465-ba90-4432-850f-322de842ccda service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:23.330533 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 676/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:26.159678 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693445a6-cdfb-4357-af20-53ede5625458 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:26.160220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693445a6-cdfb-4357-af20-53ede5625458 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:26.160220 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693445a6-cdfb-4357-af20-53ede5625458 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:26.161531 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 668/1344] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:26.166943 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213683c-02f5-469f-923e-941802950d0a None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:26.167376 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213683c-02f5-469f-923e-941802950d0a None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:26.167622 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213683c-02f5-469f-923e-941802950d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:26.198103 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-2213683c-02f5-469f-923e-941802950d0a None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:26.199317 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-2213683c-02f5-469f-923e-941802950d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:26.282352 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 677/1345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:26.319839 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693445a6-cdfb-4357-af20-53ede5625458 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:26.331211 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693445a6-cdfb-4357-af20-53ede5625458 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:26.335210 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3337-fb42-4749-92a4-9d4a9f742a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:26.335857 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3337-fb42-4749-92a4-9d4a9f742a4b service neutron] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:26.336136 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3337-fb42-4749-92a4-9d4a9f742a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:26.336406 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3337-fb42-4749-92a4-9d4a9f742a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:26.345937 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3337-fb42-4749-92a4-9d4a9f742a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:26.347383 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3337-fb42-4749-92a4-9d4a9f742a4b service neutron] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:26.357288 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 669/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:30.540886 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e69b7-7345-4faa-83c6-53ddfac7208b None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:30.541445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e69b7-7345-4faa-83c6-53ddfac7208b None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:30.541445 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e69b7-7345-4faa-83c6-53ddfac7208b None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:30.542794 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 678/1347] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:30.548205 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968ce26-7e9b-458a-bef2-aafbf862c9f8 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:30.548648 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968ce26-7e9b-458a-bef2-aafbf862c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:30.548802 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968ce26-7e9b-458a-bef2-aafbf862c9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:30.569336 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-4968ce26-7e9b-458a-bef2-aafbf862c9f8 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:30.570565 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-4968ce26-7e9b-458a-bef2-aafbf862c9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:30.580162 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 670/1348] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:30.618335 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e69b7-7345-4faa-83c6-53ddfac7208b None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:30.620170 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-fa1e69b7-7345-4faa-83c6-53ddfac7208b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:30.633708 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e69b7-7345-4faa-83c6-53ddfac7208b None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:30.637392 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48de1cd-c36c-431f-8c0a-655e661ebc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bafd4f61582941a88f68e01deb85d4a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:30.638072 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de1cd-c36c-431f-8c0a-655e661ebc0b service neutron] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:30.638350 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de1cd-c36c-431f-8c0a-655e661ebc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:30.638618 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de1cd-c36c-431f-8c0a-655e661ebc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:30.648135 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48de1cd-c36c-431f-8c0a-655e661ebc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:30.649624 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48de1cd-c36c-431f-8c0a-655e661ebc0b service neutron] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:30.659012 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 679/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:36.008445 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175eda7-e6d4-47b3-8d65-aefae1b34946 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:36.009333 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175eda7-e6d4-47b3-8d65-aefae1b34946 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:36.009763 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175eda7-e6d4-47b3-8d65-aefae1b34946 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:36.011573 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 671/1350] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:36.019005 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc25cd3-f34b-4930-bddb-00784b99a71d None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:36.019800 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc25cd3-f34b-4930-bddb-00784b99a71d None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:36.020086 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc25cd3-f34b-4930-bddb-00784b99a71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:36.150371 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-dcc25cd3-f34b-4930-bddb-00784b99a71d None None] MFA Rules not processed for user `2f5189fd89c0457da1f325be6bd6ff63`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:36.151973 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-dcc25cd3-f34b-4930-bddb-00784b99a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:36.186149 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=100 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:36.255834 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 680/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:40.558958 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c40939-369b-4eeb-8d3a-05c1cdbfc1b2 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:40.559518 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c40939-369b-4eeb-8d3a-05c1cdbfc1b2 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:40.559892 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c40939-369b-4eeb-8d3a-05c1cdbfc1b2 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:40.561447 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 672/1352] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:40.563111 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=72 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:40:40.567869 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f69e95-af53-4ac8-9088-2af14b15f373 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:40.568365 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f69e95-af53-4ac8-9088-2af14b15f373 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:40.568463 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f69e95-af53-4ac8-9088-2af14b15f373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:40.587454 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-58f69e95-af53-4ac8-9088-2af14b15f373 None None] MFA Rules not processed for user `2f5189fd89c0457da1f325be6bd6ff63`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:40.588659 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-58f69e95-af53-4ac8-9088-2af14b15f373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:40.597575 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 681/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:42.147709 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187e189-59cf-4a53-866f-2e2d72660862 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:42.148309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187e189-59cf-4a53-866f-2e2d72660862 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:42.148309 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187e189-59cf-4a53-866f-2e2d72660862 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:42.149647 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 673/1354] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:42.154719 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e70a-b32a-4965-a29a-60494d4f19f3 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:42.155147 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e70a-b32a-4965-a29a-60494d4f19f3 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:42.155414 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871e70a-b32a-4965-a29a-60494d4f19f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:42.174788 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-6871e70a-b32a-4965-a29a-60494d4f19f3 None None] MFA Rules not processed for user `2f5189fd89c0457da1f325be6bd6ff63`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:42.176081 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6871e70a-b32a-4965-a29a-60494d4f19f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:42.186513 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 682/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:46.099062 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979844af-1bff-45bd-b9d9-2563a7656303 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:46.099623 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979844af-1bff-45bd-b9d9-2563a7656303 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:46.099623 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979844af-1bff-45bd-b9d9-2563a7656303 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:46.101224 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 674/1356] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:46.107025 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ebbd-326c-4ecc-9d65-07a0008a0376 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:46.107428 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ebbd-326c-4ecc-9d65-07a0008a0376 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:46.107509 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ebbd-326c-4ecc-9d65-07a0008a0376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:46.126589 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-0b87ebbd-326c-4ecc-9d65-07a0008a0376 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:46.127792 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-0b87ebbd-326c-4ecc-9d65-07a0008a0376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:46.137016 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 683/1357] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:46.193393 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bbf8c-d24b-454e-b8eb-ae6cc6357c6f None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:46.193835 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bbf8c-d24b-454e-b8eb-ae6cc6357c6f None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:46.194239 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bbf8c-d24b-454e-b8eb-ae6cc6357c6f None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:46.195736 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 675/1358] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 14:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:46.224095 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0926-9083-4cbc-9821-2392662373fc None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:46.224456 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0926-9083-4cbc-9821-2392662373fc None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:46.225402 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0926-9083-4cbc-9821-2392662373fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:46.278243 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-26ee0926-9083-4cbc-9821-2392662373fc None None] MFA Rules not processed for user `f5c6cd73a1234dde953e73fedc6440ee`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:46.279954 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-26ee0926-9083-4cbc-9821-2392662373fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:46.365638 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 684/1359] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:46.373948 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bbf8c-d24b-454e-b8eb-ae6cc6357c6f None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:46.389053 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bbf8c-d24b-454e-b8eb-ae6cc6357c6f None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:46.394949 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fd5f0-10ca-49f3-9731-863fbeea13f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5c6cd73a1234dde953e73fedc6440ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:46.395645 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fd5f0-10ca-49f3-9731-863fbeea13f9 service nova] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:46.395930 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fd5f0-10ca-49f3-9731-863fbeea13f9 service nova] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:46.396204 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fd5f0-10ca-49f3-9731-863fbeea13f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:46.404893 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fd5f0-10ca-49f3-9731-863fbeea13f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:46.406564 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fd5f0-10ca-49f3-9731-863fbeea13f9 service nova] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:46.416870 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 676/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:48.836336 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d0292-af0e-4807-92ac-e96845b877e9 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:48.836906 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d0292-af0e-4807-92ac-e96845b877e9 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:48.836906 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d0292-af0e-4807-92ac-e96845b877e9 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:48.839176 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 685/1361] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:48.847078 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a66c-7150-408b-a086-fa3a757ff813 None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:48.847603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a66c-7150-408b-a086-fa3a757ff813 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:48.847603 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a66c-7150-408b-a086-fa3a757ff813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:48.869152 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-2a27a66c-7150-408b-a086-fa3a757ff813 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:48.870409 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-2a27a66c-7150-408b-a086-fa3a757ff813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:48.878711 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 677/1362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:48.922667 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d0292-af0e-4807-92ac-e96845b877e9 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:48.938278 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d0292-af0e-4807-92ac-e96845b877e9 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:48.943976 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4db06-1f23-4f75-8274-35531adde619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5c6cd73a1234dde953e73fedc6440ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:48.944710 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4db06-1f23-4f75-8274-35531adde619 service nova] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:48.947385 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4db06-1f23-4f75-8274-35531adde619 service nova] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:48.947385 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4db06-1f23-4f75-8274-35531adde619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:48.953956 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4db06-1f23-4f75-8274-35531adde619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:48.955612 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4db06-1f23-4f75-8274-35531adde619 service nova] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:48.969583 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 686/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:40:51.505735 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535fd9-e81d-468a-9fae-ab0b47df959a None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:51.506750 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535fd9-e81d-468a-9fae-ab0b47df959a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:51.507242 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535fd9-e81d-468a-9fae-ab0b47df959a None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:51.514451 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b163e-0fa2-4ab1-8c23-f6281cca8541 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:51.514924 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 678/1364] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:51.515891 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b163e-0fa2-4ab1-8c23-f6281cca8541 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:51.516391 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b163e-0fa2-4ab1-8c23-f6281cca8541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:51.536006 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-6e3b163e-0fa2-4ab1-8c23-f6281cca8541 None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:51.537216 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-6e3b163e-0fa2-4ab1-8c23-f6281cca8541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:51.545779 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 687/1365] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:51.589953 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ad1b7-52dc-47da-850e-988e3d1cfa64 None None] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:51.590558 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ad1b7-52dc-47da-850e-988e3d1cfa64 None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:51.591041 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ad1b7-52dc-47da-850e-988e3d1cfa64 None None] PATH_INFO: `/` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:51.592615 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 679/1366] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 14:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:51.598295 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd259-2afd-401d-9c70-a443a95c20d0 None None] REQUEST_METHOD: `POST` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:51.598676 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd259-2afd-401d-9c70-a443a95c20d0 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:51.599014 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fd259-2afd-401d-9c70-a443a95c20d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:51.618152 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.auth.core [None req-780fd259-2afd-401d-9c70-a443a95c20d0 None None] MFA Rules not processed for user `f5c6cd73a1234dde953e73fedc6440ee`. Rule list: `[]` (Enabled: `True`). {{(pid=150371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:51.619335 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.fernet_utils [None req-780fd259-2afd-401d-9c70-a443a95c20d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:51.629279 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 688/1367] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:51.637579 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ad1b7-52dc-47da-850e-988e3d1cfa64 None None] Authenticating user token {{(pid=150372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:51.652767 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ad1b7-52dc-47da-850e-988e3d1cfa64 None None] Validating token access rules against request {{(pid=150372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:51.658480 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1cc79-dacf-4900-b90a-012865b1ee48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5c6cd73a1234dde953e73fedc6440ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:51.659151 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1cc79-dacf-4900-b90a-012865b1ee48 service nova] REQUEST_METHOD: `GET` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:51.659431 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1cc79-dacf-4900-b90a-012865b1ee48 service nova] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:51.659698 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1cc79-dacf-4900-b90a-012865b1ee48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:51.668378 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1cc79-dacf-4900-b90a-012865b1ee48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:51.669968 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1cc79-dacf-4900-b90a-012865b1ee48 service nova] RBAC: Authorization granted {{(pid=150372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:51.680052 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 680/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:40:52.432247 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587b76-5a91-4bad-acd8-0a8f6ae596c7 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:52.432505 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587b76-5a91-4bad-acd8-0a8f6ae596c7 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:52.432814 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587b76-5a91-4bad-acd8-0a8f6ae596c7 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:52.434399 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 689/1369] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:40:52.440172 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05584a2-a97e-4ddb-af7b-ebb52c7b31ce None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:52.440670 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05584a2-a97e-4ddb-af7b-ebb52c7b31ce None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:52.441156 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05584a2-a97e-4ddb-af7b-ebb52c7b31ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:52.462197 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-f05584a2-a97e-4ddb-af7b-ebb52c7b31ce None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:52.463581 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-f05584a2-a97e-4ddb-af7b-ebb52c7b31ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:52.473312 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 681/1370] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:53.980477 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cc2df-2779-40b4-aa15-f43a33941aa5 None None] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:53.981062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cc2df-2779-40b4-aa15-f43a33941aa5 None None] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:53.981062 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cc2df-2779-40b4-aa15-f43a33941aa5 None None] PATH_INFO: `/` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:53.982432 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 690/1371] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:40:53.987921 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cf5ff-ba11-4611-be4d-4cad54f65a7a None None] REQUEST_METHOD: `POST` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:53.988373 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cf5ff-ba11-4611-be4d-4cad54f65a7a None None] SCRIPT_NAME: `/identity` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:53.988477 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cf5ff-ba11-4611-be4d-4cad54f65a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:54.007284 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.auth.core [None req-075cf5ff-ba11-4611-be4d-4cad54f65a7a None None] MFA Rules not processed for user `0a3ad0c94b4942efa0beedd41caa0934`. Rule list: `[]` (Enabled: `True`). {{(pid=150372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:40:54.008498 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG keystone.common.fernet_utils [None req-075cf5ff-ba11-4611-be4d-4cad54f65a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:40:54.017665 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: [pid: 150372|app: 0|req: 682/1372] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:40:54.061579 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cc2df-2779-40b4-aa15-f43a33941aa5 None None] Authenticating user token {{(pid=150371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:40:54.076826 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cc2df-2779-40b4-aa15-f43a33941aa5 None None] Validating token access rules against request {{(pid=150371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:40:54.082379 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa93e2f-0615-4b08-992a-b5e556a3be94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5c6cd73a1234dde953e73fedc6440ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e03681ba5554dc2ab23b1df5b3b4762', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:40:54.083022 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa93e2f-0615-4b08-992a-b5e556a3be94 service nova] REQUEST_METHOD: `GET` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:40:54.083328 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa93e2f-0615-4b08-992a-b5e556a3be94 service nova] SCRIPT_NAME: `/identity` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:40:54.083604 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa93e2f-0615-4b08-992a-b5e556a3be94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:40:54.091858 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa93e2f-0615-4b08-992a-b5e556a3be94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:40:54.093518 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa93e2f-0615-4b08-992a-b5e556a3be94 service nova] RBAC: Authorization granted {{(pid=150371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:40:54.103252 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: [pid: 150371|app: 0|req: 691/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:41:04.001793 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: DEBUG dbcounter [-] [150372] Writing DB stats keystone:SELECT=21 {{(pid=150372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:04.098946 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: DEBUG dbcounter [-] [150371] Writing DB stats keystone:SELECT=95 {{(pid=150371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:41:16.283694 dv-cinder-842542-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:41:16.284261 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: Thu May 19 14:41:16 2022 - graceful shutdown triggered... May 19 14:41:16.284693 dv-cinder-842542-1-iscsi devstack@keystone.service[150371]: Gracefully killing worker 1 (pid: 150371)... May 19 14:41:16.285076 dv-cinder-842542-1-iscsi devstack@keystone.service[150372]: Gracefully killing worker 2 (pid: 150372)... May 19 14:41:17.285200 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: worker 1 buried after 1 seconds May 19 14:41:17.285200 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: worker 2 buried after 1 seconds May 19 14:41:17.285200 dv-cinder-842542-1-iscsi devstack@keystone.service[150359]: goodbye to uWSGI. May 19 14:41:17.287616 dv-cinder-842542-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:41:17.288495 dv-cinder-842542-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 14:47:07.576937 dv-cinder-842542-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 14:47:07.585534 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 14:47:07.589691 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 14:47:07 2022] *** May 19 14:47:07.589849 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 14:47:07.589990 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 14:47:07.590087 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: nodename: dv-cinder-842542-1-iscsi May 19 14:47:07.590184 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: machine: x86_64 May 19 14:47:07.590293 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: clock source: unix May 19 14:47:07.590383 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: pcre jit disabled May 19 14:47:07.590482 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: detected number of CPU cores: 4 May 19 14:47:07.590578 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: current working directory: / May 19 14:47:07.590667 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: detected binary path: /usr/bin/uwsgi-core May 19 14:47:07.590761 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: your processes number limit is 47680 May 19 14:47:07.590907 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: your memory page size is 4096 bytes May 19 14:47:07.590907 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: detected max file descriptor number: 2048 May 19 14:47:07.590907 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: lock engine: pthread robust mutexes May 19 14:47:07.591189 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: thunder lock: enabled May 19 14:47:07.591256 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 14:47:07.591359 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 14:47:07.635983 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: Python main interpreter initialized at 0x55cdc369a9c0 May 19 14:47:07.635983 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: python threads support enabled May 19 14:47:07.635983 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: your server socket listen backlog is limited to 100 connections May 19 14:47:07.635983 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: your mercy for graceful operations on workers is 90 seconds May 19 14:47:07.636383 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: mapped 403077 bytes (393 KB) for 2 cores May 19 14:47:07.636383 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: *** Operational MODE: preforking *** May 19 14:47:07.636685 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: *** uWSGI is running in multiple interpreter mode *** May 19 14:47:07.636685 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: spawned uWSGI master process (pid: 213530) May 19 14:47:07.636901 dv-cinder-842542-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 14:47:07.639919 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: spawned uWSGI worker 1 (pid: 213541, cores: 1) May 19 14:47:07.639919 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: spawned uWSGI worker 2 (pid: 213542, cores: 1) May 19 14:47:07.639919 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 14:47:13.243617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:47:13.243617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: from cryptography.utils import int_from_bytes May 19 14:47:13.244521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:47:13.244521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: from cryptography.utils import int_from_bytes May 19 14:47:13.292870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:47:13.292870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: from cryptography.utils import int_from_bytes May 19 14:47:13.293557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 14:47:13.293557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: from cryptography.utils import int_from_bytes May 19 14:47:13.293675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:47:13.293675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:47:13.293812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] command line args: [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:47:13.294100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:47:13.294252 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ================================================================================ {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:47:13.294511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] admin_token = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.294757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] config_dir = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.294905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.295081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] config_source = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.295261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] debug = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.295477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.295812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.296034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] insecure_debug = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.296319 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.296571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.296769 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.296975 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_config_append = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.297147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.297320 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_dir = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.297482 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_file = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.297738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.297973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.298181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.298453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.298657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.298842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.299026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.299209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.299440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.299671 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.299902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.300142 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.300377 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.300692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] notification_format = cadf {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.300934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.301476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.301737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] publish_errors = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.301961 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.302192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.302192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.302519 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.302681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.302840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] standard_threads = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.303090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] strict_password_check = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.303301 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.303471 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] use_eventlog = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.303654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] use_journal = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.303795 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] use_json = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.303982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] use_stderr = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.304145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] use_syslog = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.304371 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] watch_log_file = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.304625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.304867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.305128 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.305397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.305648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.305888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.306136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.306374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.306620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.306864 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.307117 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.307354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.307600 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.307838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.308076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.308324 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.308561 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.308814 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.309067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.309335 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.309573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.309828 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.310092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.310348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.310597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.310832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.311063 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.311295 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.311552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.311806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.312057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.312291 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.312522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.312770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.313019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.313277 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.313531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.313765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.314002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.314239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.314469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.314700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.314932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.315163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.315399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.315647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.315878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.316115 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.316345 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.316581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.316817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.317064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.317327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.317569 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.317807 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.318051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.318288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.318523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.318753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.319003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.319239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.319475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.319705 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.319962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.320196 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.320428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.320664 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.320898 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.321163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.321429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.321678 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.321923 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.322161 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.322398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.322633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.322879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.enabled = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.323114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.323349 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.323593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.323829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.324064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.324298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.324534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.324768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.325027 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.325285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.325530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.325766 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.326007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.326245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.326480 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.326715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.326944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.327195 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.327489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.327725 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.327957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.328199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.328434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.328680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.328915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.329309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.329571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.329812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.330045 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.external = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.330276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.mapped = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.330572 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.330805 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.331036 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.password = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.331267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] auth.token = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.331505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.331742 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] catalog.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.331986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.332216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.332463 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.332710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.332943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.333179 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.333459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.333706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.333952 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.334190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.334435 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.334670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.334962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.335187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.335424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.335674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.335958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.336203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.336462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.336704 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.336959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.337193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.337455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.337699 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.337949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.338178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.338423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.338657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.338909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.339144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.339395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.339629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.339877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.340114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.340360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.340604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.340797 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 14:47:13.340931 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.341059 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 14:47:13.341187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.341358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] command line args: [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 14:47:13.341510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.341639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 14:47:13.341767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.341894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ================================================================================ {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 14:47:13.342016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.342142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] admin_token = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.342275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.342403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] config_dir = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.342531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.342658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.342786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.342914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] config_source = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.342914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] debug = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.343160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.343287 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.343443 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.343567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.343699 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.343828 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] insecure_debug = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.343954 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.344075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.344200 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.344327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.344460 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.344674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.344817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.344949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_config_append = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.344949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.345178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.345333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_dir = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.345464 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.345592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_file = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.345720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.345848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.345975 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.346109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.346231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.346358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.346488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.346609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.346746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.346868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.346868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.347294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.347429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.347558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.347694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.347816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.347816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.348057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.348057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.348312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.348446 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.348574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.348694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.348821 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.348950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.349077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.349214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] notification_format = cadf {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.349629 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.349759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.349759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.password = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.350007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.350140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.350270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] publish_errors = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.350403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.350529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.350662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.350790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.350790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.351031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.351157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.351285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.351422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.351551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.351677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] standard_threads = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.351804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.351933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] strict_password_check = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.352063 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.352190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.352311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.352437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] use_eventlog = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.352564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.352691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] use_journal = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.352691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] use_json = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.352938 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.353059 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] use_stderr = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.353178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.353325 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] use_syslog = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.353455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.353589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] watch_log_file = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 14:47:13.353710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.353835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.353962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354090 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354348 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.354965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355750 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.355889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356279 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356532 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.356789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357560 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.357935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358393 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358707 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358828 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.358955 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359082 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359349 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359861 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.359995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.360895 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] policy.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361022 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361149 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361579 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.361962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] receipt.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362083 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.362956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363582 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.363959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364451 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.364953 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] revoke.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365357 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] role.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.365978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] role.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366106 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366234 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] role.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366482 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] role.list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.366983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367233 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367627 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.367878 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368632 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.368896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369449 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369831 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369966 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.369966 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370727 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.370995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.371943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.372911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373168 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373315 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.enabled = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373824 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.373945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374195 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374576 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.374968 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375095 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375234 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375484 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.provider = fernet {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375750 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.375878 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376262 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376532 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.376916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377045 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] trust.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377478 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377605 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.377990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.378245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.378373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.378500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.378646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.378774 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.378911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379038 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379678 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.connection = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.379797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380033 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380401 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.external = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.380905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381032 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.mapped = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.381881 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.password = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382139 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] auth.token = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] catalog.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.382884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383138 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383264 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.383906 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384027 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384147 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=213542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.384524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.385550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.386588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.387607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.388607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.388607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.388607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.388607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.388607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.388607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.389299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.389299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.389299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.389654 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.389851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.390045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.390280 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.390531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.390765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.391004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.391249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.391491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.391741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.391975 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.392224 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.392472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.392709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.392956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.393203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.393460 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.393713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.393949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.password = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.394190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.394430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.394683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.394924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.395166 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.395418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.395983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.396221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.396456 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.396710 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.396961 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.397202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.397468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.397708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.398418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.398670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.398935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.399170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.399421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.399671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.399923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.400160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.400394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.400633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.400870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.401110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.401368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.401619 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.401867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.402114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.402362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.402608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.402840 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.403088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.403329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.403567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.403806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.404043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.404292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.404542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.404783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.405043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] policy.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.405323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.405574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.405813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.406050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] receipt.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.406291 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.406537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.406782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.407016 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.407255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.407496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.407745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.408680 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.408931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.409190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.409492 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.409735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] revoke.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.409988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.410118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.410257 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.410385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.410514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] role.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.410636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.410765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] role.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.410924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] role.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.411159 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] role.list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.411415 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.411667 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.411921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.412174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.412422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.412672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.412922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.413172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.413437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.413688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.413938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.414187 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.414436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.414846 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.415087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.415341 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.415600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.415851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.416102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.416340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.416594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.416849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.417091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.417349 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.417585 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.417819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.418074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.418330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.418581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.418831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.419069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.419262 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.419388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.419524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.419753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.420003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.provider = fernet {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.420242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.420498 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.420748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.421039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.421311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.421562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.421814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] trust.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.422054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.422298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.422537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.422786 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.423039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.423276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.423523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.423829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.424091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.connection = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.424337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.424586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.424827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.425071 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.425330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.425578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.425815 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.426052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.426298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.426537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.426783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.427022 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.427270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.427507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.427754 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.427988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.428242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.428479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 14:47:13.428650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=213541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 14:47:13.447358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.447592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.454055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.454292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.454524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.464647 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.483521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.483760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.500172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.521576 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.551384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.551618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.558963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.559202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.615899 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.616096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.616343 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.616561 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.616757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.616996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.617257 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.617500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.617729 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.617954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.618173 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.618394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.639955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.640176 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.640836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.640928 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641180 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641541 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.641892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.652917 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.653294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.665972 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.666209 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.666415 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.666696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.666861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.667029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.667265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=213541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.667420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.667650 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.667878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.668076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.668290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.668509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.668749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=213542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 14:47:13.684376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.684565 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:47:13.684857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.685010 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.685109 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:47:13.685193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:47:13.685406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.685539 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.685645 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 14:47:13.685733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.685811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.685880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.686106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.686242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.686346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.686675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.697600 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.697880 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.698147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.698426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.698898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.699173 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.699442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.699731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.706573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.707925 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.709868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.710145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.710420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.710810 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.711092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.711240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.711343 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.711563 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.711703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.711858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.711962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712450 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.712935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.713209 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.713502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.713772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.714041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.739000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.740464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.740598 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.740858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.742116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.742393 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.746118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.746364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.746669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.747673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.747949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.748232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.754083 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.754304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.755596 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.755875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.758900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.759184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.759462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.759737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.760003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.760278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.760497 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.760631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.760776 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.760954 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.761049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.761352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.761636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.761910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.762131 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.762432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.779792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.780062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.781567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.781845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.784682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.786454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.788332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.788622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.790109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.790408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.793635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.795436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.797422 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.799087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.801193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.801516 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.801793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.802073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.802476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.802754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.802898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.803172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.803453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.803735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.804007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.804297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.821663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.822037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.822303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.822567 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.823474 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.823855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.824132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.824398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.830604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.832332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.832476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.832555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.834170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.834425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.835644 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.835933 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.836222 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.836500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.836772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.837531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.837805 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.838093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.838389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.838663 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.850218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.850472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.850777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.851055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.851330 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.851597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.851878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.852102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.852406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.852696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.852978 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.853275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.853540 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.853814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.869920 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.873170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.875370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.876626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.878270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.879912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.880203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.880488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.881422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.884961 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.886602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.887927 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.888264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.888549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.888826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.889574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.892867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.893152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.893417 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.893719 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.896648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.898304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.901941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.902386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.902739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.903091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.903226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.903500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.903752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.911766 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=213542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.912044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.912330 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.912470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.912564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.912674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.912841 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.912984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.913122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.913398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.913661 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.913963 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.914237 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.914509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.914772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.914993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.915278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=213542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.920890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=213541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 14:47:13.921352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.921523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.921840 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.922117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.922395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.922682 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.922954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.923228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.923464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.923769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.924039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.924275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.924564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=213541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 14:47:13.953994 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cdc369a9c0 pid: 213542 (default app) May 19 14:47:13.957503 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a201eb-8762-4762-b891-383806534675 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:13.957802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a201eb-8762-4762-b891-383806534675 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:13.958073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a201eb-8762-4762-b891-383806534675 None None] PATH_INFO: `/v3` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:13.962410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Thu May 19 14:47:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:13.963353 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cdc369a9c0 pid: 213541 (default app) May 19 14:47:21.975732 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:21.976298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:21.976298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:21.978299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 2/2] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:21.988361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:21.988801 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:21.988930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:22.059215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: INFO dbcounter [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] Registered counter for database keystone May 19 14:47:22.084477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG oslo_db.sqlalchemy.engines [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=213541) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:47:22.092603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writer thread running {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:47:22.272944 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG passlib.handlers.bcrypt [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=213541) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:47:22.273331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG passlib.handlers.bcrypt [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=213541) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:47:22.329971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:22.331604 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:22.479705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 1/3] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:47:22.503562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:22.572496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: INFO dbcounter [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] Registered counter for database keystone May 19 14:47:22.594000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG oslo_db.sqlalchemy.engines [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=213542) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 14:47:22.605726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writer thread running {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 14:47:22.741001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbcc0a-cfa3-4c67-bd6f-2508ec865643 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:22.746595 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5967761725a6499fb0767a5715d5e41f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:22.747273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:22.747558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:22.747834 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:22.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:22.808743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:47:22.809085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba532fe0-b31e-44a7-9b2b-a814b84686c4 admin admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:22.819833 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 3/4] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 14:47:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:47:22.827356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:22.849008 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db3e35-dd47-4320-9d89-9e154393b5ec None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:22.855563 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5967761725a6499fb0767a5715d5e41f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:22.856705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:22.856984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:22.857279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] PATH_INFO: `/v3/projects/5967761725a6499fb0767a5715d5e41f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:22.859140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] RBAC: Authorizing `identity:get_project(project_id=5967761725a6499fb0767a5715d5e41f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:22.911405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 14:47:22.911711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af6094-cae9-49bf-a884-892e7613c96f admin admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:22.916361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 2/5] 10.222.0.73 () {56 vars in 1207 bytes} [Thu May 19 14:47:22 2022] GET /identity/v3/projects/5967761725a6499fb0767a5715d5e41f => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:25.079699 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1598a0b-86a2-477e-b11c-01f8e5b792d4 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:25.080580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1598a0b-86a2-477e-b11c-01f8e5b792d4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:25.080580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1598a0b-86a2-477e-b11c-01f8e5b792d4 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:25.087654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 4/6] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:25.093528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:25.093984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:25.094098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:25.114910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:25.116419 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:25.125810 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 3/7] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 14:47:25.148615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1598a0b-86a2-477e-b11c-01f8e5b792d4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:25.171801 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1598a0b-86a2-477e-b11c-01f8e5b792d4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:25.177474 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174912b-e3e6-4a65-afad-0b185995e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5967761725a6499fb0767a5715d5e41f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:25.178106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174912b-e3e6-4a65-afad-0b185995e2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:25.178391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174912b-e3e6-4a65-afad-0b185995e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:25.178663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174912b-e3e6-4a65-afad-0b185995e2e2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:25.187023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174912b-e3e6-4a65-afad-0b185995e2e2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:25.188930 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174912b-e3e6-4a65-afad-0b185995e2e2 admin admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:25.197970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 5/8] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 14:47:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:47:25.203223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:25.212265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0bc4a-9da9-4eb4-8621-7b3448149561 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:25.217148 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581d375-5625-4f4c-b2bb-78035ec21669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5967761725a6499fb0767a5715d5e41f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:25.218057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581d375-5625-4f4c-b2bb-78035ec21669 admin admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:25.218339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581d375-5625-4f4c-b2bb-78035ec21669 admin admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:25.218610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581d375-5625-4f4c-b2bb-78035ec21669 admin admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:25.220052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581d375-5625-4f4c-b2bb-78035ec21669 admin admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:25.221894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581d375-5625-4f4c-b2bb-78035ec21669 admin admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:25.223457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 4/9] 10.222.0.73 () {56 vars in 1198 bytes} [Thu May 19 14:47:25 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:27.502966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65021dc-a348-479a-b80e-f208d4a7f29a None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:27.503544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65021dc-a348-479a-b80e-f208d4a7f29a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:27.503544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65021dc-a348-479a-b80e-f208d4a7f29a None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:27.504407 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4ff9-74d8-41ea-a661-8fcfb183989e None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:27.504783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4ff9-74d8-41ea-a661-8fcfb183989e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:27.504971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4ff9-74d8-41ea-a661-8fcfb183989e None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:27.505806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 6/10] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:27.509262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 5/11] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:27.511509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:27.511853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:27.512199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:27.514849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:27.515289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:27.515577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:27.533026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:27.534271 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:27.552336 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG passlib.handlers.bcrypt [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=213542) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 14:47:27.552750 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG passlib.handlers.bcrypt [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=213542) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 14:47:27.598590 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:27.600156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:27.620004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:47:27.642548 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:27.669133 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:27.702151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:27.725400 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:27.752491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06223cf7-678c-422a-b9a9-c1baa31d6063 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:27.789690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:27.790284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:27.790568 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:27.790845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:27.945360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8873-360e-472c-895e-b47fb2ace419 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:27.947723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:27.982893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:27.983423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:27.984552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:27.984684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:27.984840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:27.985116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:27.997654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:28.007325 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:28.009106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:28.010199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:28.011444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 8/14] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:47:28.016245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:28.086132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 7/15] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:28.091721 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:28.118072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17387593-3e80-4fef-a0dd-259316f4bbe5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:28.152493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:28.153172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:28.153486 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:28.153764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:28.154530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:28.154840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:28.156798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:28.168912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 9/16] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:28.174655 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:28.190257 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847ad3-dccc-4851-a629-16029cc6aba7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:28.205557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:28.218132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:28.232111 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:28.232931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:28.233211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:28.233515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:28.244379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:28.245916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:47:28.266685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:28.267696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:28.268862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:47:28.268862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:47:28.268862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:47:28.268862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:28.268862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 8/17] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:47:28.276214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:28.288274 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470da799-3d52-4050-b6cc-5de7bd00def4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:28.324042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:28.324937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:28.325221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:28.325525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:28.327143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:28.329020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:28.330379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 10/18] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:28 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:28.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:28.376645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bd7c4-72b6-4461-baa7-9a731061bce3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:28.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:28.410565 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:28.410846 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:28.411118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:28.411851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:28.412161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:28.413591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:28.430454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 9/19] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:28.430999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:28.438259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2765f-99da-43d4-b9ae-d0121f3862b9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:28.477072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7987a2-5e5d-46f4-8a82-cd016aa1fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:28.477919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7987a2-5e5d-46f4-8a82-cd016aa1fbba None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:28.478194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7987a2-5e5d-46f4-8a82-cd016aa1fbba None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:28.478460 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7987a2-5e5d-46f4-8a82-cd016aa1fbba None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:28.479740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7987a2-5e5d-46f4-8a82-cd016aa1fbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:28.481208 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7987a2-5e5d-46f4-8a82-cd016aa1fbba None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:28.483115 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 11/20] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:28.489351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:28.531857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:28.608327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415e86f-2549-4532-aa03-0acb1bc0ade5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:28.650316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:28.651004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:28.651290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:28.651562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:28.652421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.domain.id=default, user.id=f7af2cbac96a492e94720a0ed454d0be) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:28.652735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:28.654240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1044ab-be8d-47f1-9ccd-bfe61f1ba7d0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:28.661331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 10/21] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 14:47:28 2022] GET /identity/v3/role_assignments?user.id=f7af2cbac96a492e94720a0ed454d0be&scope.domain.id=default&role.id=d450d657419649a6ae78b9e8578202ca => generated 233 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:31.127423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f70cf-857a-4fd4-8d9c-17d461abf22f None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.127959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f70cf-857a-4fd4-8d9c-17d461abf22f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.127959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f70cf-857a-4fd4-8d9c-17d461abf22f None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.129586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 12/22] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:31.134799 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.135212 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.135399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.162313 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:31.163574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:31.217145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a59bf-e3fc-4f15-bcef-ee14ca4c48f2 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.217507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a59bf-e3fc-4f15-bcef-ee14ca4c48f2 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.217718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a59bf-e3fc-4f15-bcef-ee14ca4c48f2 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.219153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 13/23] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:47:31.228533 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.228787 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.229026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.250187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:31.251409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:31.287477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 11/24] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:31.357713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:31.376017 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 14/25] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 14:47:31.405172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:31.421669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:31.528887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:31.540651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8696d-bebd-4d41-a89e-2e804f1fff69 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:31.583935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:31.584757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.585038 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.585333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.616703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305c431-47f6-47a3-b07d-0cc8bf0279b3 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:31.634166 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:31.641984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:47:31.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:31.675726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:31.676647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:47:31.676647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:47:31.676647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:47:31.676647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:31.676647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 15/26] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:47:31.687831 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:31.715361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537764b-8e23-4bc6-97ff-09b7d6c6d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:31.716193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537764b-8e23-4bc6-97ff-09b7d6c6d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.716479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537764b-8e23-4bc6-97ff-09b7d6c6d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.716757 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537764b-8e23-4bc6-97ff-09b7d6c6d8f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.743647 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537764b-8e23-4bc6-97ff-09b7d6c6d8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9537764b-8e23-4bc6-97ff-09b7d6c6d8f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:31.755531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:31.756519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:31.757793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 12/27] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:31.758519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91645-7d70-42f6-b74b-eac39f05a49b None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.758519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91645-7d70-42f6-b74b-eac39f05a49b None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.758659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91645-7d70-42f6-b74b-eac39f05a49b None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.760880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 13/28] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:31.761808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224e26f-2c6a-44ac-bb7b-1b96168a03d7 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.762093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224e26f-2c6a-44ac-bb7b-1b96168a03d7 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.762374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224e26f-2c6a-44ac-bb7b-1b96168a03d7 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.763773 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 14/29] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:31.768135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.768495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.768781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.790851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:31.792333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:31.840846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b1694-4e87-4229-b1a5-0214ca04dc17 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:31.879163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 15/30] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:31.880389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:31.885515 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:31.887175 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.889580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.890206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.891159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:31.891677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:31.893316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031830-7815-407a-b80e-fcc73965f992 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:31.917772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:31.918103 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 16/31] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:31.924746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:31.925365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:31.925853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:31.933100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:31.947019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:31.948554 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:32.017709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 17/32] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:32.025317 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f155883-1fc9-4cc4-a8d8-1a3c5acc2a7e None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.025978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.064093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.064780 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.065065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.065387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.066160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:32.066457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.067867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:32.085969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 16/33] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:32.086693 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.153682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e30a88-9691-4fbd-a859-00078a16df44 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.174093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:32.191759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.192775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.193317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.193799 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.194763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.194996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fbdee-3c9d-4e5b-bdc8-58701b327ee3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.196662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:32.199391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:32.231575 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.232250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.232533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.232810 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.233670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.235126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:32.333549 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 18/34] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:47:32 2022] POST /identity/v3/roles => generated 219 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:32.334779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.395979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 17/35] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:47:32 2022] POST /identity/v3/projects => generated 301 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:32.395979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.514125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52288f1f-84f8-49f1-a21b-30d8631c0f7a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.581533 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f19e-8d5a-4e71-b8c8-66f18dd7421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.582433 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f19e-8d5a-4e71-b8c8-66f18dd7421a None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.582824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f19e-8d5a-4e71-b8c8-66f18dd7421a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.583082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51f8d-d5f4-48a2-8b03-8d8fb6ad5b3e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.583297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f19e-8d5a-4e71-b8c8-66f18dd7421a None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.584267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f19e-8d5a-4e71-b8c8-66f18dd7421a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.586109 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f19e-8d5a-4e71-b8c8-66f18dd7421a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:32.610224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 19/36] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:47:32 2022] POST /identity/v3/roles => generated 225 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:32.617307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cbdbb-e681-4fc4-9feb-2815a18b7a2d None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.617307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cbdbb-e681-4fc4-9feb-2815a18b7a2d None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.617307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cbdbb-e681-4fc4-9feb-2815a18b7a2d None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.617307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 20/37] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:32.637532 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21c4bb-ad6a-446b-a4de-36a56a507032 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.638201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21c4bb-ad6a-446b-a4de-36a56a507032 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.638731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21c4bb-ad6a-446b-a4de-36a56a507032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.648190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:32.659390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.660892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.663448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.664078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.665976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.670890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:32.673818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-8e21c4bb-ad6a-446b-a4de-36a56a507032 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:32.674492 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 18/38] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:32 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:32.676267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-8e21c4bb-ad6a-446b-a4de-36a56a507032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:32.680626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.757969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:32.789954 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 21/39] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:47:32.821621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f54b-d4b1-4e66-bddb-d5356959ef2e None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.822178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f54b-d4b1-4e66-bddb-d5356959ef2e None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.822710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f54b-d4b1-4e66-bddb-d5356959ef2e None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.825068 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 22/40] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:47:32.826494 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389f54b-d4b1-4e66-bddb-d5356959ef2e None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.838445 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8299d38-f138-4639-b241-58040a8910b0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.879908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d10fd4-1194-447e-8f17-4c7119746236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.880777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10fd4-1194-447e-8f17-4c7119746236 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.881064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10fd4-1194-447e-8f17-4c7119746236 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.881369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10fd4-1194-447e-8f17-4c7119746236 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.882684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d10fd4-1194-447e-8f17-4c7119746236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.884167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d10fd4-1194-447e-8f17-4c7119746236 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:32.885923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 19/41] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:32.887318 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.887612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.887889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.907677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:32.909221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:32.949402 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389f54b-d4b1-4e66-bddb-d5356959ef2e None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:32.973912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:32.980590 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 20/42] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:32.981502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:32.985585 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:32.986960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:32.986960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:32.986960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:32.987667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:32.989090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:33.004933 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 23/43] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:47:32 2022] POST /identity/v3/roles => generated 223 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:33.014612 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:33.068393 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:33.103506 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357fb14-03c2-44a9-b8a8-0f5320fac72c None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:33.136565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00809-ef09-412c-8f83-e2e955c4d854 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:33.141937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:33.142870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:33.143155 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:33.143439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] PATH_INFO: `/v3/domains/default/users/f7af2cbac96a492e94720a0ed454d0be/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:33.146558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f7af2cbac96a492e94720a0ed454d0be, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:33.148827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:33.165395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 21/44] 10.222.0.73 () {58 vars in 1361 bytes} [Thu May 19 14:47:32 2022] PUT /identity/v3/domains/default/users/f7af2cbac96a492e94720a0ed454d0be/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:47:33.188744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5fb3a-9a58-4c4a-b5a7-5bacaf72e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:33.189606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5fb3a-9a58-4c4a-b5a7-5bacaf72e362 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:33.190601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5fb3a-9a58-4c4a-b5a7-5bacaf72e362 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:33.190893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5fb3a-9a58-4c4a-b5a7-5bacaf72e362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:33.192312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5fb3a-9a58-4c4a-b5a7-5bacaf72e362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:33.193888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5fb3a-9a58-4c4a-b5a7-5bacaf72e362 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:33.195631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 24/45] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:33.201841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:33.272926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:33.321154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99095776-c02b-4986-9c7a-505b510d552b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:33.358247 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db2571-f543-4d4e-b1d4-5c4ab34b5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:33.358721 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2571-f543-4d4e-b1d4-5c4ab34b5839 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:33.358981 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2571-f543-4d4e-b1d4-5c4ab34b5839 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:33.359276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2571-f543-4d4e-b1d4-5c4ab34b5839 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:33.359996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2571-f543-4d4e-b1d4-5c4ab34b5839 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:33.361486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2571-f543-4d4e-b1d4-5c4ab34b5839 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:33.389110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 22/46] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:47:33 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:35.369478 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926835-708b-421a-a91e-c85a371bbd9c None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.370049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926835-708b-421a-a91e-c85a371bbd9c None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.370049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926835-708b-421a-a91e-c85a371bbd9c None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.371511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 25/47] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:35.377273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d357d0f-0093-4422-9632-f8312234d1d5 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.377706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d357d0f-0093-4422-9632-f8312234d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.377970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d357d0f-0093-4422-9632-f8312234d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.396436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-4d357d0f-0093-4422-9632-f8312234d1d5 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:35.397704 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-4d357d0f-0093-4422-9632-f8312234d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:35.472959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 23/48] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:35.496227 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926835-708b-421a-a91e-c85a371bbd9c None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:35.588810 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239522b6-c103-48d1-a9cf-1de08dfbe64a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.589258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239522b6-c103-48d1-a9cf-1de08dfbe64a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.589586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239522b6-c103-48d1-a9cf-1de08dfbe64a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.590978 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 24/49] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:35.596142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.596430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.596717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.603303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926835-708b-421a-a91e-c85a371bbd9c None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:35.613836 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:35.615053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:35.617158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:35.643835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:35.644652 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.644937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.645215 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.650880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:35.656135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:35.663896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:35.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:35.665755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 26/50] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:35.671241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:35.688202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 25/51] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:47:35.711108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:35.776487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84466329-f99f-4ead-84d4-ebbc1c9ea5a2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:35.809144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09e73b-cdfb-4dac-9f1a-e204aee7e9df None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:35.812708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:35.813425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.813712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.813988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.814752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:35.815061 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:35.816472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:35.824600 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 27/52] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:35.830206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:35.843942 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:35.844752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.845028 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.845338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.846552 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:35.848041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:35.849777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 26/53] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:35.855728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:35.865821 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:35.896848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:35.929944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeb7a3-66d7-4767-ac5e-5fb0b7bb573a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:35.956358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbbfaf-1f73-465e-a07d-22e121432147 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:35.963855 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:35.964765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.965055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.965366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.966643 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:35.968183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:35.969956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 28/54] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:35 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:35.976584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:35.991674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:35.992352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:35.992638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:35.992915 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:35.993671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:35.995241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:36.024811 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 27/55] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 14:47:35 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:36.086701 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b4019-e0be-4791-8e17-0fa08b617d97 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:36.088522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:36.126755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729596d0-c29e-4916-ba56-f669f8fc3c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:36.127577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729596d0-c29e-4916-ba56-f669f8fc3c9a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:36.127850 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729596d0-c29e-4916-ba56-f669f8fc3c9a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:36.128154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729596d0-c29e-4916-ba56-f669f8fc3c9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:36.129524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729596d0-c29e-4916-ba56-f669f8fc3c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:36.131006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729596d0-c29e-4916-ba56-f669f8fc3c9a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:36.132695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 29/56] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:36.138994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:36.237399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:36.251326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad24791-fa64-41db-9997-bdf861e0db84 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:36.286690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:36.287424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:36.287755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:36.288094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:36.289158 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.domain.id=default, user.id=f7af2cbac96a492e94720a0ed454d0be) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:36.289575 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:36.291318 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ba4b-cfeb-4d14-9b0a-0e8f78d9be99 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:36.298126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 28/57] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 14:47:36 2022] GET /identity/v3/role_assignments?user.id=f7af2cbac96a492e94720a0ed454d0be&scope.domain.id=default&role.id=d450d657419649a6ae78b9e8578202ca => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:38.244168 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8851-2fc7-49e9-b76e-8991a3ae3346 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:38.245049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8851-2fc7-49e9-b76e-8991a3ae3346 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:38.245049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8851-2fc7-49e9-b76e-8991a3ae3346 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:38.247281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 30/58] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:38.256202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:38.256634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:38.256881 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:38.277894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:38.279152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:38.353948 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 29/59] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:38.379191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea8851-2fc7-49e9-b76e-8991a3ae3346 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:38.495206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea8851-2fc7-49e9-b76e-8991a3ae3346 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:38.544000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498c9f2-b831-4670-b263-f4b84330e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:38.544794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498c9f2-b831-4670-b263-f4b84330e97a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:38.545078 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498c9f2-b831-4670-b263-f4b84330e97a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:38.545387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498c9f2-b831-4670-b263-f4b84330e97a None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:38.546724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498c9f2-b831-4670-b263-f4b84330e97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:38.548179 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498c9f2-b831-4670-b263-f4b84330e97a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:38.549819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 31/60] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:38.558606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:38.647921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:38.666605 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11c555-6b43-4104-8800-a9882eced6b9 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:38.701756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2837897-d14a-4895-b1b9-fe904d22b8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:38.702520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2837897-d14a-4895-b1b9-fe904d22b8f2 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:38.702806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2837897-d14a-4895-b1b9-fe904d22b8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:38.703084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2837897-d14a-4895-b1b9-fe904d22b8f2 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:38.703805 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2837897-d14a-4895-b1b9-fe904d22b8f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:38.705283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2837897-d14a-4895-b1b9-fe904d22b8f2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:38.732524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 30/61] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:47:38 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:41.915955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd455-61d4-4447-b739-0a69d643b0bb None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:41.916984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd455-61d4-4447-b739-0a69d643b0bb None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:41.917576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd455-61d4-4447-b739-0a69d643b0bb None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:41.919196 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 32/62] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:41.930686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c340dbd-c112-4a4b-a940-559d66e51032 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:41.931557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c340dbd-c112-4a4b-a940-559d66e51032 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:41.932062 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c340dbd-c112-4a4b-a940-559d66e51032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:41.956064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-2c340dbd-c112-4a4b-a940-559d66e51032 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:41.957607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-2c340dbd-c112-4a4b-a940-559d66e51032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:42.055944 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 31/63] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:42.080152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd455-61d4-4447-b739-0a69d643b0bb None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.152243 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.236927 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd455-61d4-4447-b739-0a69d643b0bb None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.255232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983328e-1d94-4ec5-aba6-776f7ef71207 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.255584 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983328e-1d94-4ec5-aba6-776f7ef71207 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.255802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983328e-1d94-4ec5-aba6-776f7ef71207 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.257424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 32/64] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:42.279334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd2490-f6ed-4d31-8d18-6fb263fc1df2 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.280125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd2490-f6ed-4d31-8d18-6fb263fc1df2 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.280709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd2490-f6ed-4d31-8d18-6fb263fc1df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.302130 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bca6c-ce55-4057-bd60-979e2d3119ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.303374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bca6c-ce55-4057-bd60-979e2d3119ed None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.303885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bca6c-ce55-4057-bd60-979e2d3119ed None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.304380 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bca6c-ce55-4057-bd60-979e2d3119ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.306018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-c1bd2490-f6ed-4d31-8d18-6fb263fc1df2 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:42.307656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-c1bd2490-f6ed-4d31-8d18-6fb263fc1df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:42.320565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bca6c-ce55-4057-bd60-979e2d3119ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-d12bca6c-ce55-4057-bd60-979e2d3119ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:42.328416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:42.329626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:42.330674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:42.332762 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 33/65] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:42.334186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8cac9-e9a9-4f9c-8b75-9f2808b8fad9 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.334605 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8cac9-e9a9-4f9c-8b75-9f2808b8fad9 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.335030 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8cac9-e9a9-4f9c-8b75-9f2808b8fad9 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.336428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 34/66] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:42.341191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8cac9-e9a9-4f9c-8b75-9f2808b8fad9 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.390920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.396900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 33/67] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:42.398539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.398936 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.399321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.429515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:42.430740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:42.464607 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.470917 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8cac9-e9a9-4f9c-8b75-9f2808b8fad9 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.505257 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 34/68] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:42.506195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.512777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.513583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.513947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.514300 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.515346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:42.515729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:42.517360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:42.527939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 35/69] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:42.533524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.609125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc105f-2848-47d8-be61-d3dff171074d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.638317 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.644580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68c5f9-336f-419e-82b5-74e4f41ab998 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.649049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.650149 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.650589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.651032 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.664033 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:42.673213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:42.674298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:42.675190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 35/70] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:42.677533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.688404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe2a40-487e-4fb9-ac67-5b593fddf8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.689378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2a40-487e-4fb9-ac67-5b593fddf8ee None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.689669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2a40-487e-4fb9-ac67-5b593fddf8ee None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.689952 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2a40-487e-4fb9-ac67-5b593fddf8ee None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.707666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe2a40-487e-4fb9-ac67-5b593fddf8ee None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:42.709424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe2a40-487e-4fb9-ac67-5b593fddf8ee None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:42.711341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 36/71] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:42.712740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e4d78-c626-41cc-9d0c-5ba6a53ad99f None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.713191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e4d78-c626-41cc-9d0c-5ba6a53ad99f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.713645 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e4d78-c626-41cc-9d0c-5ba6a53ad99f None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.715048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 37/72] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:42.719232 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e4d78-c626-41cc-9d0c-5ba6a53ad99f None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.739113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.793739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110c766-db7f-4901-b3f2-433fcb9cc614 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.829664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.830488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.830781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.831042 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.832434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e4d78-c626-41cc-9d0c-5ba6a53ad99f None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.842704 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:42.851437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:42.852423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 36/73] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:42.853755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.870247 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.870944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.871230 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.871511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.872275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:42.872583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:42.874060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c10a7c-894e-4c1b-8bac-0fde08e28876 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:42.882091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 38/74] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:42.886679 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cd89-4891-4020-980e-489631ddd542 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.886972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cd89-4891-4020-980e-489631ddd542 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.887253 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cd89-4891-4020-980e-489631ddd542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.903797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-ef33cd89-4891-4020-980e-489631ddd542 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:42.903968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.905502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-ef33cd89-4891-4020-980e-489631ddd542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:42.954657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7519-5068-433a-b402-d3e4960cc622 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:42.971515 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 39/75] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:42.972819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09cc8-413a-4acd-a4cf-3aad6450c051 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.973109 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09cc8-413a-4acd-a4cf-3aad6450c051 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.973431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09cc8-413a-4acd-a4cf-3aad6450c051 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:42.974690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 40/76] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:42.975394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b09cc8-413a-4acd-a4cf-3aad6450c051 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:42.989261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:42.997283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf301ee-40a8-40eb-bf9d-479b5963f14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:42.997957 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf301ee-40a8-40eb-bf9d-479b5963f14c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:42.998246 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf301ee-40a8-40eb-bf9d-479b5963f14c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:42.998527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf301ee-40a8-40eb-bf9d-479b5963f14c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.067525 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf301ee-40a8-40eb-bf9d-479b5963f14c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.069055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf301ee-40a8-40eb-bf9d-479b5963f14c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.081476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b09cc8-413a-4acd-a4cf-3aad6450c051 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.118742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.120475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.121188 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.121596 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.121880 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.122682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:43.122999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.124462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.132884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 41/77] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:43.133802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.171474 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 37/78] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:43.173038 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.173352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.173634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.189791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:43.190990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:43.206404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.235955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ccb4-df98-4ae3-8aa5-4d8a80ae85b9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.256896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 38/79] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:43.257864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.271076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.271986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.272272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.272549 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.273853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.275326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.276969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 42/80] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:43.277770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.342336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.361990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86121da-e505-4985-97a0-c3d1f1924e75 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.376387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bdb00-0944-4259-b4dc-7d0249d3fd21 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.394648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.395460 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.396056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.396334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.407023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:43.411539 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.412432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.412714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.412986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.414355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:43.414868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:43.415852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:43.416733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:43.417830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.417917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 39/81] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:43.417917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.422059 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 43/82] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:43.422888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.434946 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.520957 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef57a3-7b33-45be-ad77-a6ea47ac3cc5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.525282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68f77f-491a-47b3-8681-bc24211f4d08 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.553875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.554761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.555051 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.555327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.556576 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.558069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.558584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.559241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.559526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.559798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 40/83] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:43.559902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.560571 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.571316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.633820 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.635290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.645337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.659729 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23737132-b25f-4f34-8f47-8795a4688288 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.694546 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.695354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.695636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.695908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.697088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.698556 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.700187 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 41/84] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:43.702044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.734146 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 44/85] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:43.735008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.788054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.812605 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a1972-57fd-4799-a788-0e79b53afc00 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.834530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04542673-d22b-461a-8a35-b0a136e2a336 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.843672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:43.846840 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.847570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.847889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.848166 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.848922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] RBAC: Adding query filter params (name=member) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:43.849228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.850662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.858875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 42/86] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:43.859660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:43.870823 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.871475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.871760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.872006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.939800 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:43.941213 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:43.961613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcb44b-f263-4441-ae47-2bdf3626f921 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:43.996171 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:43.996830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:43.997117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:43.997383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:43.998260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=497304e37c85429c8c3148acfed14451) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:43.998563 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.000079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.006165 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 43/87] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/role_assignments?user.id=497304e37c85429c8c3148acfed14451&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:44.006572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.009163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.051271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 45/88] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:44.051271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.063307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.131110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ac7ce-8611-4a07-9679-d1852840dcf3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.170035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.170804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.171102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.171386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.172119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.173708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.178565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219803dd-42ec-47f1-9c18-a932ca17af73 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.196465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 44/89] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:47:44 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:44.202457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.218692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.219608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.219893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.220172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.221511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.223015 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.224992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 46/90] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:44.225680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.279276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.332211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f61d8-8ade-4f98-84fd-46a1fc95a459 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.342640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.372005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.372979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.373325 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.373611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.375007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.375635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49e8a-68bc-4461-a0e0-9a9233ca4447 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.376559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.378802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 45/91] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:44.379930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.420120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.420748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.421027 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.421384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.422666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.424256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.426241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 47/92] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:44.430019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.502899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad894e45-8c9b-44e5-a1ee-12959725686e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.532427 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.548660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.549806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.550337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.550830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.551903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] RBAC: Adding query filter params (role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=497304e37c85429c8c3148acfed14451) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:44.552422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.554350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.560953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 46/93] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/role_assignments?user.id=497304e37c85429c8c3148acfed14451&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=75add8e0c0454b3a8681eab45a26244c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:44.562134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.562616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86e86b-9670-4753-ae6d-dde40dfe5c37 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.602620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.603316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.603597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.603869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.620663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.683340 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.685404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.714517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbcd26-8228-42c3-a272-cbfa0eb94a39 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:44.760075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:44.761123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:44.761739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:44.762330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:44.763382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=f7af2cbac96a492e94720a0ed454d0be) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:44.763904 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:44.765698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:44.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 47/94] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/role_assignments?user.id=f7af2cbac96a492e94720a0ed454d0be&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:44.807307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 48/95] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:44.813108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:44.925481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:44.976032 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919010-2302-420e-bac6-66f5906eb957 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:45.021007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589064a-0621-47ad-bd39-d05571a46c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:45.021850 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589064a-0621-47ad-bd39-d05571a46c88 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:45.022129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589064a-0621-47ad-bd39-d05571a46c88 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:45.022401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589064a-0621-47ad-bd39-d05571a46c88 None admin] PATH_INFO: `/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:45.023606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589064a-0621-47ad-bd39-d05571a46c88 None admin] RBAC: Authorizing `identity:get_project(project_id=be14f43897c0487c99a98cc0f2a5ef0b)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:45.025076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589064a-0621-47ad-bd39-d05571a46c88 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:45.027164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 48/96] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:44 2022] GET /identity/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:45.033304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:45.080644 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:45.137184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d4aa1-f0e7-487c-9f98-ca20c822f049 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:45.173018 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:45.173758 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:45.174039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:45.174314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:45.175192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=be14f43897c0487c99a98cc0f2a5ef0b, user.id=497304e37c85429c8c3148acfed14451) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:45.175501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:45.177017 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b46ff-2b3e-4e09-b567-cc3169ce6f71 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:45.193427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 49/97] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:45 2022] GET /identity/v3/role_assignments?user.id=497304e37c85429c8c3148acfed14451&scope.project.id=be14f43897c0487c99a98cc0f2a5ef0b&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:46.975677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4c675-d6b3-43ad-9228-7d5f594b8283 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:46.976215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4c675-d6b3-43ad-9228-7d5f594b8283 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:46.976215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4c675-d6b3-43ad-9228-7d5f594b8283 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:46.977859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 49/98] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:46.983185 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:46.983637 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:46.983747 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.001198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:47.002472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:47.070084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 50/99] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:47.104919 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4c675-d6b3-43ad-9228-7d5f594b8283 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:47.234109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4c675-d6b3-43ad-9228-7d5f594b8283 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:47.263082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:47.282749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296bc3d-cb0f-40a9-a7d2-d8cc0f0d70d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:47.283567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296bc3d-cb0f-40a9-a7d2-d8cc0f0d70d6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.283849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296bc3d-cb0f-40a9-a7d2-d8cc0f0d70d6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.284122 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296bc3d-cb0f-40a9-a7d2-d8cc0f0d70d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.299216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296bc3d-cb0f-40a9-a7d2-d8cc0f0d70d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-b296bc3d-cb0f-40a9-a7d2-d8cc0f0d70d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:47.307153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:47.308125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:47.309717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:47.311763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 50/100] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:47.317157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:47.365709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:47.386333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9d77a-a2a7-4672-994f-ea74743fb815 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.386617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9d77a-a2a7-4672-994f-ea74743fb815 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.386896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9d77a-a2a7-4672-994f-ea74743fb815 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.388285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 51/101] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:47:47.393521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.393867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.394148 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.415868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:47.419976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:47.478764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4041d-d236-4395-80d4-13b37ef2b7b0 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:47.548962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 52/102] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:47.550832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:47.551749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.552237 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.552739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.553835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:47.554346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:47.555973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:47.565095 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 51/103] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:47.574350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:47.594786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:47.752057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:47.756785 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d603af-37cc-451f-baf8-2b166e5ef3d1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:47.761848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53fee9-f5c4-4a04-bb80-e6ad79d7daf6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:47.792167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:47.799140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703b2ee-9929-4896-b294-8cd55e6efe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:47.800544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703b2ee-9929-4896-b294-8cd55e6efe4c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.801039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703b2ee-9929-4896-b294-8cd55e6efe4c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.801569 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703b2ee-9929-4896-b294-8cd55e6efe4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.820366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703b2ee-9929-4896-b294-8cd55e6efe4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-f703b2ee-9929-4896-b294-8cd55e6efe4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:47.832127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:47.833692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:47:47.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:47.837073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 52/104] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:47:47.838626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de230d57-a6fe-4876-8256-b695a4ad914b None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.839137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de230d57-a6fe-4876-8256-b695a4ad914b None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.839677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de230d57-a6fe-4876-8256-b695a4ad914b None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.841228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 53/105] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:47.847395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de230d57-a6fe-4876-8256-b695a4ad914b None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:47.853982 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce42fb-ebec-4b4e-894e-656933694e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:47.854908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce42fb-ebec-4b4e-894e-656933694e63 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.855201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce42fb-ebec-4b4e-894e-656933694e63 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.855481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce42fb-ebec-4b4e-894e-656933694e63 None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.856771 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce42fb-ebec-4b4e-894e-656933694e63 None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:47.858433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce42fb-ebec-4b4e-894e-656933694e63 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:47.860113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 53/106] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:47 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:47.861292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:47.861586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:47.861864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:47.897791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:47.900921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:48.024272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 54/107] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:48.025261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.028872 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de230d57-a6fe-4876-8256-b695a4ad914b None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.093760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.094425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.094713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.094988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.095751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:48.096054 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.100455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.117397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 54/108] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:48.118280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.166869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:48.224118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb442d-abc4-497c-90eb-89d8d446e6b8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.245867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:48.259035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa12508-4a6b-4cb0-8c7e-18f3583fdcf8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.271327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.272249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.272862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.273148 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.274756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.276794 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.279030 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 55/109] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:48 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:48.280181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.304619 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb6e8d-83cc-496d-a194-e7d9559e850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.305535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb6e8d-83cc-496d-a194-e7d9559e850b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.305824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb6e8d-83cc-496d-a194-e7d9559e850b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.306107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb6e8d-83cc-496d-a194-e7d9559e850b None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.308100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb6e8d-83cc-496d-a194-e7d9559e850b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.310186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb6e8d-83cc-496d-a194-e7d9559e850b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.313043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 55/110] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:48.318848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424b6e5-9b43-4f4e-97cc-f4c295187e48 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.319190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424b6e5-9b43-4f4e-97cc-f4c295187e48 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.319422 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424b6e5-9b43-4f4e-97cc-f4c295187e48 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.321081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 56/111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:48.326008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424b6e5-9b43-4f4e-97cc-f4c295187e48 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.410364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94c64c-4053-472a-8b94-bb9b0622480d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.436821 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:48.446052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.446965 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.447252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.447529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] PATH_INFO: `/v3/users/33f8958da4ee434d8c1624649dffd616` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.449374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424b6e5-9b43-4f4e-97cc-f4c295187e48 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.463676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] RBAC: Authorizing `identity:get_user(user_id=33f8958da4ee434d8c1624649dffd616)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.465479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.467270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 56/112] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:48 2022] GET /identity/v3/users/33f8958da4ee434d8c1624649dffd616 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:48.467270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.503289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ce594-ec54-403d-839c-ce6f86f70903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.505507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ce594-ec54-403d-839c-ce6f86f70903 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.505507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ce594-ec54-403d-839c-ce6f86f70903 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.505507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ce594-ec54-403d-839c-ce6f86f70903 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/497304e37c85429c8c3148acfed14451/roles/e7782d74fe1a4e02b57e8e7377ba7e89` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.509405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ce594-ec54-403d-839c-ce6f86f70903 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f218d2e4a774028b7261264ea3fdd59, user_id=497304e37c85429c8c3148acfed14451, role_id=e7782d74fe1a4e02b57e8e7377ba7e89)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.510088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:48.510947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ce594-ec54-403d-839c-ce6f86f70903 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.525919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 57/113] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:47:48 2022] PUT /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/497304e37c85429c8c3148acfed14451/roles/e7782d74fe1a4e02b57e8e7377ba7e89 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:47:48.537565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d6e4-a753-472b-8c6b-8b8ee781bbde None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.537565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d6e4-a753-472b-8c6b-8b8ee781bbde None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.537565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d6e4-a753-472b-8c6b-8b8ee781bbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.551757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-8106d6e4-a753-472b-8c6b-8b8ee781bbde None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:48.553002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-8106d6e4-a753-472b-8c6b-8b8ee781bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:48.585655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534bcb-99eb-46b8-a576-afc27ea350c4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.630626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0e0c7-3006-45f5-bbf4-187a8f489d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.631330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0e0c7-3006-45f5-bbf4-187a8f489d5f None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.631674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0e0c7-3006-45f5-bbf4-187a8f489d5f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.631963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0e0c7-3006-45f5-bbf4-187a8f489d5f None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.640360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 58/114] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:48.640590 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0e0c7-3006-45f5-bbf4-187a8f489d5f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.640688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5316-4f10-4c47-854e-05b49a75a4e4 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.640688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5316-4f10-4c47-854e-05b49a75a4e4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.640853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0e0c7-3006-45f5-bbf4-187a8f489d5f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.640934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5316-4f10-4c47-854e-05b49a75a4e4 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.640934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 59/115] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:48.640934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea5316-4f10-4c47-854e-05b49a75a4e4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.680655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 57/116] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:47:48 2022] POST /identity/v3/projects => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:48.682735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e937566-89b5-427e-8bde-32506e1f33b5 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.682735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e937566-89b5-427e-8bde-32506e1f33b5 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.682735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e937566-89b5-427e-8bde-32506e1f33b5 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.684066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 58/117] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:48.693381 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.693381 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.693381 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.730052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:48.731906 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:48.753948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:48.785955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea5316-4f10-4c47-854e-05b49a75a4e4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:48.825266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 59/118] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:48.826195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:48.845639 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:48.846814 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:48.847349 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:48.847905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:48.868522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:48.942006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:48.943440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:48.970614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fe079-3e34-479b-8195-2b4318090bd4 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.012354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78eef6-cbe6-48b8-8069-66f229248fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.013773 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78eef6-cbe6-48b8-8069-66f229248fd8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.014335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78eef6-cbe6-48b8-8069-66f229248fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.014826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78eef6-cbe6-48b8-8069-66f229248fd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.026947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78eef6-cbe6-48b8-8069-66f229248fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-eb78eef6-cbe6-48b8-8069-66f229248fd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:49.035951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:49.036933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:49.037866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:49.040339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 60/119] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:49.041976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.042491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.042983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.057459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.061688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:49.063162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:49.085696 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 60/120] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:49.089868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.180307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 61/121] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:49.180307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.185773 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.226935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa820-08b0-4eb6-9831-a209b1576ff7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.263730 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.264996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.265536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.266051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.277665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:49.286935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:49.287966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:49.288884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:49.291450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 61/122] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:49.292606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.313266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5910ddf-3bc9-4fb9-9cc9-22db145075a1 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.347710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.352170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.353094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.353966 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.354723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.356174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] RBAC: Adding query filter params (name=member) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:49.356765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:49.358447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:49.368592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 62/123] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:49.369747 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.432580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1d8ef-869f-4846-9cf7-6dc9257d8e8c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.464167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.468996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.469948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.470234 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.470512 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.471751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:49.473268 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:49.479987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 62/124] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:49.480784 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.502848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12373645-33f9-4202-a931-1e6aad479dbd None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.540947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.542184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.542698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.543169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.555126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:49.564470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:49.565731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:49.566834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:49.569055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 63/125] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:49.570012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.583266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa817b75-726f-4a54-b87c-828ed87c0f9f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.592439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.617836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.618529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.618818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.619098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.619950 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:49.620167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:49.621748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:49.637709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 63/126] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:49.638753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.686472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c72f-2e9a-4c6b-98bd-5fbfff96a46f None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.719303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.726945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.728049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.728520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.730118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.731520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:49.733647 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:49.735796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 64/127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:49.736840 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.748089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99a346-5aa2-4fdc-bd65-82f1929c944d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.783402 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.784073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.784358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.784634 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.785507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] RBAC: Adding query filter params (role.id=e59278c8302b4d63a8f546cccc1ac139, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=33f8958da4ee434d8c1624649dffd616) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:49.785820 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:49.787666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:49.793069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 64/128] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/role_assignments?user.id=33f8958da4ee434d8c1624649dffd616&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=e59278c8302b4d63a8f546cccc1ac139 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:49.802976 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:49.879352 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:49.907575 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572db6e-8629-4555-8f22-1f925916d8be None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:49.945564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:49.946645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:49.947101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:49.947555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:49.948522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:49.949104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:49.950936 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:49.961035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 65/129] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:49.962350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.008196 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e85c-ea4b-4a45-ba51-0421bde2e97c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.052968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:50.056942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.057930 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.058789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.059072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.060312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.062031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.063991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 65/130] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:50.064600 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.124488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63d349-ab36-4039-bc69-28d5345da657 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.189746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.190638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.190920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.191194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] PATH_INFO: `/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.192501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] RBAC: Authorizing `identity:get_project(project_id=be14f43897c0487c99a98cc0f2a5ef0b)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.193999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.195734 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 66/131] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:49 2022] GET /identity/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:50.196495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.284677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b47b32-d1cd-4eb8-bc02-bf9ff069759b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.289548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:50.321421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86506e19-d799-4fd9-9324-315a5d76fa98 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.356677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:50.364026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.366711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.367571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.368553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.370882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.372853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.374317 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.375452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.377351 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 66/132] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:50 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:50.377856 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.378331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.379747 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.380755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.382814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.385080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 67/133] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:50 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:50.389798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.521425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda1384-f825-49a4-982b-32bbb0241448 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.557223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.558067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.558354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.558630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.559831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.561400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.563122 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 68/134] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:50 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:50.563866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.611238 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd7223-35d9-40a8-a46d-2b4026a7dc52 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.633048 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:50.673637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5669cc5-bf0c-4837-8d82-028cb10569c5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.708109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.709028 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.709346 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.709628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/f7af2cbac96a492e94720a0ed454d0be/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.711645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] RBAC: Authorizing `identity:create_grant(project_id=6f218d2e4a774028b7261264ea3fdd59, user_id=f7af2cbac96a492e94720a0ed454d0be, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.713119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.731024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 69/135] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:47:50 2022] PUT /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/f7af2cbac96a492e94720a0ed454d0be/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:47:50.731896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:50.732020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f52af-1897-4abf-a11f-9ef46a6a8041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.732906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f52af-1897-4abf-a11f-9ef46a6a8041 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.733285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f52af-1897-4abf-a11f-9ef46a6a8041 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.733571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f52af-1897-4abf-a11f-9ef46a6a8041 None admin] PATH_INFO: `/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b/users/497304e37c85429c8c3148acfed14451/roles/e7782d74fe1a4e02b57e8e7377ba7e89` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.746183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f52af-1897-4abf-a11f-9ef46a6a8041 None admin] RBAC: Authorizing `identity:create_grant(project_id=be14f43897c0487c99a98cc0f2a5ef0b, user_id=497304e37c85429c8c3148acfed14451, role_id=e7782d74fe1a4e02b57e8e7377ba7e89)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.747681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f52af-1897-4abf-a11f-9ef46a6a8041 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.812265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 67/136] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:47:50 2022] PUT /identity/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b/users/497304e37c85429c8c3148acfed14451/roles/e7782d74fe1a4e02b57e8e7377ba7e89 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:47:50.901267 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d72898-dac8-4dda-9a90-77766c32301b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:50.933878 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:50.942462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e82dfa-14a9-4f22-a67c-7536d287297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:50.943635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e82dfa-14a9-4f22-a67c-7536d287297c None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:50.944117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e82dfa-14a9-4f22-a67c-7536d287297c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:50.944413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e82dfa-14a9-4f22-a67c-7536d287297c None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/497304e37c85429c8c3148acfed14451/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:50.947618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e82dfa-14a9-4f22-a67c-7536d287297c None admin] RBAC: Authorizing `identity:create_grant(project_id=6f218d2e4a774028b7261264ea3fdd59, user_id=497304e37c85429c8c3148acfed14451, role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:50.949215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e82dfa-14a9-4f22-a67c-7536d287297c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:50.978227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 70/137] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:47:50 2022] PUT /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/497304e37c85429c8c3148acfed14451/roles/75add8e0c0454b3a8681eab45a26244c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:47:51.242027 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a69216-e992-4ff2-b9bf-6bcf259f76ab None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.242581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a69216-e992-4ff2-b9bf-6bcf259f76ab None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.242581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a69216-e992-4ff2-b9bf-6bcf259f76ab None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.243956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 68/138] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:51.251129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.251670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.251670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.254685 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33d8f3-94bb-4b4d-97c1-987cd6024a32 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.254971 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33d8f3-94bb-4b4d-97c1-987cd6024a32 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.255254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33d8f3-94bb-4b4d-97c1-987cd6024a32 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.256929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 69/139] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:51.264557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.264852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.265513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.281031 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:51.282274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:51.302158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:51.303362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:51.335284 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:51.379758 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 70/140] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:47:51.410854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:51.454931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 71/141] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:51.494553 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:51.549749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49f22e-836b-4775-8883-898ae9a92857 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:51.585647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:51.586467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.586748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.587023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.606576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:51.624969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:51.626069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:51.626947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 71/142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:47:51.634544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:51.669683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:51.743218 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc248-cf78-44fd-9bc5-0f5ad8ae2b1d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:51.769335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:51.769974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf1c46-cc56-489d-aa66-e453d433c63f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:51.818321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:51.819689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.820988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.822154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.824819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:51.826789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:51.830660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 72/143] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:51.838116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:51.846135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:51.847935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:51.848445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:51.849794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:51.850819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:51.851140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:51.852612 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:51.865568 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 72/144] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:51.868705 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:51.953530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:51.976032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b7be-9312-408c-b5a0-49e21353378c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:52.048359 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:52.049186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.049491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.049765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.054116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:52.056024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:52.058341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 73/145] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:51 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:52.063753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:52.066263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f4e43-60f6-4f43-8792-d4bf040a9e35 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:52.143357 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:52.144057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.144337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.144597 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.145336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:52.147122 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:52.192834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 73/146] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:47:51 2022] POST /identity/v3/users => generated 283 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:52.257393 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22722c56-d930-41bf-9ecf-275ad0079f2e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:52.307631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:52.308276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.308577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.308854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.335198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:52.399642 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:52.401125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:52.539093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 74/147] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:47:52.546391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:52.614772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:52.689653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738f196-04eb-4dc9-bebe-fe814007d150 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:52.725077 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0c1d2-776b-4f55-ae15-89b4dab48095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:52.726300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0c1d2-776b-4f55-ae15-89b4dab48095 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.726770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0c1d2-776b-4f55-ae15-89b4dab48095 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.727194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0c1d2-776b-4f55-ae15-89b4dab48095 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.728689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0c1d2-776b-4f55-ae15-89b4dab48095 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:52.730399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0c1d2-776b-4f55-ae15-89b4dab48095 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:52.732384 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 74/148] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:52 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:52.742414 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:52.798890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:52.847993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca266-7c09-4722-83d5-acd777d4b50a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.848344 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca266-7c09-4722-83d5-acd777d4b50a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.848558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca266-7c09-4722-83d5-acd777d4b50a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.850215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 75/149] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:52.858036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.858036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.858036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.868648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6aaba-72e9-4de1-ae0e-ff40a90368d1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:52.876435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:52.879189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:52.908601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:52.909711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:52.910246 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:52.910781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:52.912073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=497304e37c85429c8c3148acfed14451) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:52.912793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:52.914572 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:52.925351 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 75/150] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:52 2022] GET /identity/v3/role_assignments?user.id=497304e37c85429c8c3148acfed14451&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:52.953858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 76/151] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:47:52.976869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:53.105495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1e08-8814-4aef-8b69-27667eef57c4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:53.121364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:53.144064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:53.144906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:53.145186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:53.145496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:53.157888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:53.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:53.167055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:53.167944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 76/152] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:53.175378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:53.213026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:53.280781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c3f25-2071-49f0-b7b2-1a6f74e11d22 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:53.315842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:53.316529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:53.316812 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:53.317083 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:53.317872 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:53.318177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:53.319587 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:53.328553 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 77/153] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:53.334338 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:53.441104 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dde51-b0df-404f-8f28-4c153b61d03d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:53.476141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:53.477051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:53.477359 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:53.477636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] PATH_INFO: `/v3/users/33f8958da4ee434d8c1624649dffd616` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:53.478874 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] RBAC: Authorizing `identity:get_user(user_id=33f8958da4ee434d8c1624649dffd616)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:53.480365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:53.486675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 77/154] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:53 2022] GET /identity/v3/users/33f8958da4ee434d8c1624649dffd616 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:53.488065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:53.595493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:53.602199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a1e54-2d96-453c-a60c-808943f7c417 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:53.665994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e049127-3dd0-4091-911e-5bc456404d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:53.666906 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e049127-3dd0-4091-911e-5bc456404d55 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:53.667190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e049127-3dd0-4091-911e-5bc456404d55 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:53.667472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e049127-3dd0-4091-911e-5bc456404d55 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:53.668816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e049127-3dd0-4091-911e-5bc456404d55 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:53.670484 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e049127-3dd0-4091-911e-5bc456404d55 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:53.673655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 78/155] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:53 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:53.678582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:53.730892 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:53.795305 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44803f-6413-4c90-9ff6-0cea36119038 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:53.830764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5751-6ad2-4527-968c-e668d3d1b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:53.831751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5751-6ad2-4527-968c-e668d3d1b2e6 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:53.832037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5751-6ad2-4527-968c-e668d3d1b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:53.832315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5751-6ad2-4527-968c-e668d3d1b2e6 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/33f8958da4ee434d8c1624649dffd616/roles/e59278c8302b4d63a8f546cccc1ac139` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:53.834432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5751-6ad2-4527-968c-e668d3d1b2e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f218d2e4a774028b7261264ea3fdd59, user_id=33f8958da4ee434d8c1624649dffd616, role_id=e59278c8302b4d63a8f546cccc1ac139)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:53.835898 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5751-6ad2-4527-968c-e668d3d1b2e6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:53.857677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 78/156] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:47:53 2022] PUT /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59/users/33f8958da4ee434d8c1624649dffd616/roles/e59278c8302b4d63a8f546cccc1ac139 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:47:54.415119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147eb2b-ac81-4d0a-99cf-54ca7421e854 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.415533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147eb2b-ac81-4d0a-99cf-54ca7421e854 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.415721 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147eb2b-ac81-4d0a-99cf-54ca7421e854 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.417410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 79/157] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:54.423904 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f133a8-9f3c-4904-a96e-74284e3f8006 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.424817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f133a8-9f3c-4904-a96e-74284e3f8006 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.425383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f133a8-9f3c-4904-a96e-74284e3f8006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.447631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-12f133a8-9f3c-4904-a96e-74284e3f8006 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:54.449337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-12f133a8-9f3c-4904-a96e-74284e3f8006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:54.471649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d0e5-42ca-465f-8207-cb676439db4a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.472275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d0e5-42ca-465f-8207-cb676439db4a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.472778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d0e5-42ca-465f-8207-cb676439db4a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.474430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 80/158] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:47:54.479410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.479973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.480532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.509386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:54.512785 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:54.559942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 79/159] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:54.561731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4425d7-f7f3-4296-b7b0-e24f01eb15ee None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.562198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4425d7-f7f3-4296-b7b0-e24f01eb15ee None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.562638 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4425d7-f7f3-4296-b7b0-e24f01eb15ee None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.565948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 80/160] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:54.570086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.570540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.570998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.593318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:54.594756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:54.622506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:54.657816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 81/161] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:54.658860 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:54.720607 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 81/162] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:54.721424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:54.828271 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25640f1-7a81-4cd3-a029-5cbbfb86f34f None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:54.836228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:54.863192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:54.863999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:54.864628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:54.865205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:54.876398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:54.884504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:54.885722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:47:54.886726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:54.888791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 82/163] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:54.889786 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:54.963599 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa8aaa-3bbc-4077-803d-07bc2a8fd033 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.020443 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.021880 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.022880 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.023469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.025441 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f91d08-f173-47d8-9917-e6192cd62dab None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.046640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:55.054842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:55.056016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:55.057167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:55.065428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 82/164] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:55.069317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.096056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.098703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.099141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.099578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.119101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:55.125471 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:55.129611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:55.130745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:55.131644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:47:55.134541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 83/165] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:47:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:55.135720 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.162438 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:55.271484 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3147a-b016-46d6-830d-0431bd1f3ce8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.279365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921ecd-b86c-4bdb-8661-7b24505c3210 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.335613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.336659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.337586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.338312 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.339282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:55.340026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:55.342158 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:55.355793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.356624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.356977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.357479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.358398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:55.358906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:55.360401 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:55.360885 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 84/166] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:55.361976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.383757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 83/167] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:55.384848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.483413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:55.506808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d9016-82c5-454d-88d2-a8d0624aea6a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.542173 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.542910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.543193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.543464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.544213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:55.544522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:55.546089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:55.552674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:55.556380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 85/168] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:55.557203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.585848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3a45f-0df3-46b3-a75e-59de8abe4dc6 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.650555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.651476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.651764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.652047 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] PATH_INFO: `/v3/users/f7af2cbac96a492e94720a0ed454d0be` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.657970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] RBAC: Authorizing `identity:get_user(user_id=f7af2cbac96a492e94720a0ed454d0be)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:55.659464 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:55.659579 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fb1ef-fe85-493e-a0f7-9b0ae7ecb4f3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.661193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 84/169] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/users/f7af2cbac96a492e94720a0ed454d0be => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:55.662084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.699071 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.699968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.700251 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.700556 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.702013 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:55.703506 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:55.705751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 86/170] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:55.706343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.733600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:55.816207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3023c-723e-406a-a15a-e6be418a2214 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.862262 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.863167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.863449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.863724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] PATH_INFO: `/v3/users/497304e37c85429c8c3148acfed14451` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:55.865674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] RBAC: Authorizing `identity:get_user(user_id=497304e37c85429c8c3148acfed14451)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:55.867144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:55.870388 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 85/171] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/users/497304e37c85429c8c3148acfed14451 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:55.871472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:55.909269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:55.925299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab42d4d-5925-49a0-92af-a8674ab54e0c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:55.987468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:55.988124 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:55.988400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:55.988666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.056992 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.058568 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.064245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7daa1-08c5-4cc4-8498-7641520e2c15 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:56.070255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:56.126276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:56.127538 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:56.128047 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:56.128522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.158469 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 87/172] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:56.159156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.248822 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.250864 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.261602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d18d3-fc03-4841-b57f-3fca23efd51a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:56.273569 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:56.289596 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:56.297154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:56.297847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:56.298127 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:56.298399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.380683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.382337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.438128 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 86/173] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:56.439009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.502194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 88/174] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:56.503868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.516874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:56.582147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b43fa-c556-4884-908e-6d3ffbbede0e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:56.606344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:56.625794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:56.628256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:56.628927 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:56.630234 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.635935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.637690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.639754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 87/175] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:56.640960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.702450 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062602b-1e77-4621-a191-f073451f2de8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:56.774074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:56.774972 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:56.775250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:56.775522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] PATH_INFO: `/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.776801 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] RBAC: Authorizing `identity:get_project(project_id=be14f43897c0487c99a98cc0f2a5ef0b)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.778295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.780067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 89/176] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/projects/be14f43897c0487c99a98cc0f2a5ef0b => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:56.780835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.801022 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997a7f2-ef7f-49d2-a8c1-147e79fe35d5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:56.843031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:56.844131 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:56.844420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:56.844698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.845965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.847548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.849436 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 88/177] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:56.850218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.859625 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:56.883292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719aca-16af-4c87-a7f9-84773f638b7c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:56.918064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:56.919009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:56.919448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:56.919876 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:56.921007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=f7af2cbac96a492e94720a0ed454d0be) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:56.921524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:56.923174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:56.929453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 90/178] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/role_assignments?user.id=f7af2cbac96a492e94720a0ed454d0be&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:56.930476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:56.956969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:57.057284 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed913ce-9fcd-4482-aba8-8a8326af9e9b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.082847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa162e-b97a-481d-817c-f26e7e26963b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.123718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.124782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.125955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.126433 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.127535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=be14f43897c0487c99a98cc0f2a5ef0b, user.id=497304e37c85429c8c3148acfed14451) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:57.128241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:57.131273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7c5b8-9d89-44e7-8e00-383d19d6ae21 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:57.143817 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.144894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.145410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.146208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.147264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] RBAC: Adding query filter params (role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=497304e37c85429c8c3148acfed14451) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:57.147787 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:57.148020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 89/179] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/role_assignments?user.id=497304e37c85429c8c3148acfed14451&scope.project.id=be14f43897c0487c99a98cc0f2a5ef0b&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:57.149747 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5c2fb-2508-4440-b621-f9a2f12e4a9b None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.151564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c5a5d-be66-4f16-ac87-8e3996014d2a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:57.152141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5c2fb-2508-4440-b621-f9a2f12e4a9b None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.154921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5c2fb-2508-4440-b621-f9a2f12e4a9b None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.156833 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 90/180] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:57.161947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 91/181] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:56 2022] GET /identity/v3/role_assignments?user.id=497304e37c85429c8c3148acfed14451&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=75add8e0c0454b3a8681eab45a26244c => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:57.163683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.164168 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.164657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.187884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:57.189537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:57.270998 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893503f-b4f6-497e-9fca-3b5a781a7d9e None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.271670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893503f-b4f6-497e-9fca-3b5a781a7d9e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.272146 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893503f-b4f6-497e-9fca-3b5a781a7d9e None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.273864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 92/182] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:47:57.286834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2593-aa8b-4e5f-b83a-ebbc320d0bb9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.287480 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2593-aa8b-4e5f-b83a-ebbc320d0bb9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.287954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2593-aa8b-4e5f-b83a-ebbc320d0bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.290249 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 91/183] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:57.301377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:57.320165 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.321608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-aa1e2593-aa8b-4e5f-b83a-ebbc320d0bb9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:57.323005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-aa1e2593-aa8b-4e5f-b83a-ebbc320d0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:57.384685 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:57.439482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 93/184] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:57.442113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1e26f-c61b-4651-85aa-73ec76a7019d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.442719 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1e26f-c61b-4651-85aa-73ec76a7019d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.443216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1e26f-c61b-4651-85aa-73ec76a7019d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.444853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 94/185] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:57.450807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.451440 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.452026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.452740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5b327-7a75-46d7-946d-406b54c72aad None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.471411 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:57.472985 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:57.496930 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.497825 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.498305 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.498768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.513465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:57.524996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:57.526382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:47:57.527523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:57.529824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 92/186] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:57.530865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.577272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 95/187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:47:57.577272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.664214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:57.693556 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6885511-6bbc-4051-839b-a8fbe7dde52e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.705813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a85916-e11b-4e36-ba48-a935eced94a3 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.706733 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:57.735875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90703572-a227-416e-9654-88998f8044f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.736707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90703572-a227-416e-9654-88998f8044f8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.737096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90703572-a227-416e-9654-88998f8044f8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.737405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90703572-a227-416e-9654-88998f8044f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.743330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.744014 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.744297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.744572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.745558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:57.746155 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:57.747571 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:57.748557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90703572-a227-416e-9654-88998f8044f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:57.756665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 96/188] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-90703572-a227-416e-9654-88998f8044f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:57.756884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:57.758065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:57.759001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 93/189] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:47:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:57.761748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.763077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90703572-a227-416e-9654-88998f8044f8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.860018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527567-922f-4ff0-98a6-1694772b67d9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.860304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90703572-a227-416e-9654-88998f8044f8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:57.885493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:57.892558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884599-ca7a-4bc5-a999-7dda32b382c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.893458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884599-ca7a-4bc5-a999-7dda32b382c9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.893732 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884599-ca7a-4bc5-a999-7dda32b382c9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.893985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884599-ca7a-4bc5-a999-7dda32b382c9 None admin] PATH_INFO: `/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.894390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:57.895192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.895472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.896058 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.897262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:57.898757 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:57.900363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 97/190] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:47:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:57.901121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.909764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1884599-ca7a-4bc5-a999-7dda32b382c9 None admin] RBAC: Authorizing `identity:get_user(user_id=9cb6c32718a34f96bb2b80c0468dcdc2)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:57.911268 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1884599-ca7a-4bc5-a999-7dda32b382c9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:57.912959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 94/191] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:57 2022] GET /identity/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:57.914171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bae67-078c-4ff3-b471-04dd3fa3a374 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:57.914452 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bae67-078c-4ff3-b471-04dd3fa3a374 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:57.914747 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bae67-078c-4ff3-b471-04dd3fa3a374 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:57.916127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 95/192] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:47:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:47:57.916620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bae67-078c-4ff3-b471-04dd3fa3a374 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:57.943979 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.007770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4ebe-406c-4577-96fa-dfd74ffd6851 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.015221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bae67-078c-4ff3-b471-04dd3fa3a374 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.042942 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.043613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.043894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.044166 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.044916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:58.045223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.046842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.051145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f357547-ba18-435c-8ea8-3de97ed3b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.051813 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357547-ba18-435c-8ea8-3de97ed3b648 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.052093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357547-ba18-435c-8ea8-3de97ed3b648 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.052371 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357547-ba18-435c-8ea8-3de97ed3b648 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.053095 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f357547-ba18-435c-8ea8-3de97ed3b648 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.054560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f357547-ba18-435c-8ea8-3de97ed3b648 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.055043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 98/193] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:47:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:58.055739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.083846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 96/194] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:47:57 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:47:58.085122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.085447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.085718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.111246 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:47:58.112467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:47:58.115471 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.165998 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7979e-4f4a-4e1a-a5a1-8c1e5e34bae1 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.197890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 97/195] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:47:58.197890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.211255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.211951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.212238 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.212516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.220485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.291912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.292558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.320704 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84409546-3fbb-4634-b6b3-4b52ed930248 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.363655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.364186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.365489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.365997 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.366251 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] PATH_INFO: `/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.368230 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] RBAC: Authorizing `identity:get_user(user_id=9cb6c32718a34f96bb2b80c0468dcdc2)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.370260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.372582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 98/196] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:58.373702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.431784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 99/197] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:58.432661 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.486928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.507054 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb546f-2f81-4073-9bb0-d0320e75640c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.549356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07fb0d-cdd2-4677-8581-21f02fb2de3e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.550945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.552124 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.552643 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.553131 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.564919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:47:58.573552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:47:58.574723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:47:58.575618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:47:58.578663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 99/198] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:47:58.584218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.586302 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.586967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.587250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.587528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.617425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.664542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.666201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.699258 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357600f-bf70-4a86-b893-2be101285f43 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.742134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.744450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.744945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.745472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.746923 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.748714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.756495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 100/199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:58.756495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.775881 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.784452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 100/200] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:58.786031 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.868877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:58.877756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81960607-8219-4716-8d0f-da5ca2a4311f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.915327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.916302 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.916739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.917334 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.918364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:58.918840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.920572 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.924311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ee674-cc10-48b5-817a-d9afb8686771 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:58.929947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 101/201] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:58.930961 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:58.964210 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:58.964794 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:58.965073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:58.965373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:58.966210 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=9cb6c32718a34f96bb2b80c0468dcdc2) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:58.966522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:58.968075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:58.974061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 101/202] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/role_assignments?user.id=9cb6c32718a34f96bb2b80c0468dcdc2&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:58.974891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:59.010217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:59.047429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f7ffd-11a6-4ef8-b9aa-09ab4bea9096 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:59.085686 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:59.086888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:59.087327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:59.087754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:59.089213 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:59.090969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:59.093434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 102/203] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:59.099320 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:59.160448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365fa69-c8a9-42fa-aeb8-5dd3056fb07a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:59.189746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:59.205055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:59.206700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:59.206987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:59.207259 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] PATH_INFO: `/v3/users/33f8958da4ee434d8c1624649dffd616` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:59.208724 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] RBAC: Authorizing `identity:get_user(user_id=33f8958da4ee434d8c1624649dffd616)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:59.210890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:59.217295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 102/204] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:47:58 2022] GET /identity/v3/users/33f8958da4ee434d8c1624649dffd616 => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:59.220138 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:59.221422 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81265d87-32dc-41cd-8f7e-9b2422bf91dd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:59.281425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:59.282810 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:59.283313 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:59.283818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:59.284995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] RBAC: Adding query filter params (role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=9cb6c32718a34f96bb2b80c0468dcdc2) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:59.285583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:59.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:59.294386 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 103/205] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:59 2022] GET /identity/v3/role_assignments?user.id=9cb6c32718a34f96bb2b80c0468dcdc2&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=75add8e0c0454b3a8681eab45a26244c => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:47:59.343540 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666851c2-81a5-4735-a332-6ffabcb18c53 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:59.382011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:59.382676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:59.382957 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:59.383227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:59.449912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:59.461627 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:59.463843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:59.571987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 103/206] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:47:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:47:59.577729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:59.584684 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:59.735257 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70806b-6e80-4bef-abbc-21c05528bf30 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:59.771963 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f5d2a-7d60-4b9d-9675-f8dca194a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:59.772851 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f5d2a-7d60-4b9d-9675-f8dca194a71b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:59.773133 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f5d2a-7d60-4b9d-9675-f8dca194a71b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:59.773440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f5d2a-7d60-4b9d-9675-f8dca194a71b None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:59.774657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f5d2a-7d60-4b9d-9675-f8dca194a71b None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:59.776121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f5d2a-7d60-4b9d-9675-f8dca194a71b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:59.778278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 104/207] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:47:59 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:47:59.784655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:47:59.900994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:47:59.917425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ad479-bde2-4809-b1cb-4a325fc2a32b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:47:59.953451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:47:59.954147 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:47:59.954424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:47:59.954691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:47:59.955526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] RBAC: Adding query filter params (role.id=e59278c8302b4d63a8f546cccc1ac139, scope.project.id=6f218d2e4a774028b7261264ea3fdd59, user.id=33f8958da4ee434d8c1624649dffd616) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:47:59.955833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:47:59.957308 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614afc17-08be-408e-a3db-0b046ce4134a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:47:59.963378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 104/208] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:47:59 2022] GET /identity/v3/role_assignments?user.id=33f8958da4ee434d8c1624649dffd616&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=e59278c8302b4d63a8f546cccc1ac139 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:00.608065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974acf7-f5f4-48f4-b2e1-6abc963394b4 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:00.608385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974acf7-f5f4-48f4-b2e1-6abc963394b4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:00.608786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974acf7-f5f4-48f4-b2e1-6abc963394b4 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:00.610284 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 105/209] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:00.626030 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:00.626667 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:00.627158 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:00.646637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:00.648038 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:00.714510 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 105/210] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:00.739426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974acf7-f5f4-48f4-b2e1-6abc963394b4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:00.825346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:00.863394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974acf7-f5f4-48f4-b2e1-6abc963394b4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:00.899324 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9238ccc-6096-497c-8490-f5968df9918c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:00.900137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238ccc-6096-497c-8490-f5968df9918c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:00.900422 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238ccc-6096-497c-8490-f5968df9918c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:00.900697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238ccc-6096-497c-8490-f5968df9918c None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:00.901916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9238ccc-6096-497c-8490-f5968df9918c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:00.903369 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9238ccc-6096-497c-8490-f5968df9918c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:00.905143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 106/211] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:00.910771 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:00.962993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa895ea-2558-4b6b-833d-4cf288ffeadb None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:00.963319 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa895ea-2558-4b6b-833d-4cf288ffeadb None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:00.963551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa895ea-2558-4b6b-833d-4cf288ffeadb None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:00.965085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 107/212] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:00.970295 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:00.970580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:00.970868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:00.988789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:00.990092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:01.014209 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:01.022646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955671b6-a54d-4679-9b57-2b76f879aba8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:01.065345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:01.066421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.066867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.067293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.068158 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:01.069471 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 108/213] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:01.069983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:01.095163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 106/214] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:00 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:01.100818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:01.183436 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:01.218615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152112d-5815-4e96-9b40-704f5cdb8b83 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:01.257177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:01.258339 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.258785 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.259212 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.272068 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:01.280675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:01.282074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:01.283293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:01.285622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 109/215] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:01.291791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:01.410982 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d386a-a978-45c6-86c0-204a3ec78017 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:01.451956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:01.452577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.452860 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.453277 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.454054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] RBAC: Adding query filter params (name=member) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:01.455438 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:01.455765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c5a3-89e5-4f0b-ab5a-38c77ef0ec87 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:01.467558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:01.467852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 107/216] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:01.474045 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:01.591446 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61595104-a695-424a-bdc9-536f8f2b93b8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:01.602933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1595eb2-727d-4567-b5de-5c4946464770 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.603269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1595eb2-727d-4567-b5de-5c4946464770 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.603499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1595eb2-727d-4567-b5de-5c4946464770 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.605834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 108/217] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:01.619061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.619360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.619630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.644016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:01.659412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:01.661547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.662072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.662618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] PATH_INFO: `/v3/users/e3eac434204841d49bc8601968cb7f74` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.667079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:01.667547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:01.679273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] RBAC: Authorizing `identity:get_user(user_id=e3eac434204841d49bc8601968cb7f74)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:01.681091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:01.683392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 110/218] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:01 2022] GET /identity/v3/users/e3eac434204841d49bc8601968cb7f74 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:01.693086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:01.822893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 109/219] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:01.845184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:01.924718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fd378-713c-4614-91af-09bb9b152604 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:01.957371 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc383d7-06e2-492d-b078-6d441e41ac7d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:01.973275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735834a9-7317-4eff-9edd-efeef9846876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:01.973275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735834a9-7317-4eff-9edd-efeef9846876 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:01.973275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735834a9-7317-4eff-9edd-efeef9846876 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:01.973275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735834a9-7317-4eff-9edd-efeef9846876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:01.996216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.002624 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3d27c-d484-42dd-98ad-79ec9753061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.003854 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3d27c-d484-42dd-98ad-79ec9753061c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.004248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3d27c-d484-42dd-98ad-79ec9753061c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.004617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3d27c-d484-42dd-98ad-79ec9753061c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.016953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3d27c-d484-42dd-98ad-79ec9753061c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-4ec3d27c-d484-42dd-98ad-79ec9753061c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:02.026275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:02.027499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:02.028487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:02.033541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 110/220] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:02.035061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29986760-73c9-4d59-9493-9887696a5969 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.035425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29986760-73c9-4d59-9493-9887696a5969 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.035787 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29986760-73c9-4d59-9493-9887696a5969 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.037684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 111/221] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:02.038320 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29986760-73c9-4d59-9493-9887696a5969 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.049028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.064492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735834a9-7317-4eff-9edd-efeef9846876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.066354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735834a9-7317-4eff-9edd-efeef9846876 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.150864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29986760-73c9-4d59-9493-9887696a5969 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.179444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 111/222] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:02.181133 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8390e-ed25-42df-997e-017af436550c None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.181656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8390e-ed25-42df-997e-017af436550c None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.182161 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8390e-ed25-42df-997e-017af436550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.187891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.189472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.189472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.189472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.190226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:02.190603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.192087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.201201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 112/223] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:02.201550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-83c8390e-ed25-42df-997e-017af436550c None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:02.203006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-83c8390e-ed25-42df-997e-017af436550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:02.207175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.264858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.280727 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 112/224] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:02.282769 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8390e-ed25-42df-997e-017af436550c None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.303629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.319026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a2730-e1b4-4357-971a-ce0358c436ea None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.355886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.357125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.357544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.358759 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.360459 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.362373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.369883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 113/225] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:02.369883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.405266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8390e-ed25-42df-997e-017af436550c None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.448056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.449404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.449679 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.449957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] PATH_INFO: `/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.451195 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] RBAC: Authorizing `identity:get_user(user_id=9cb6c32718a34f96bb2b80c0468dcdc2)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.453064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.455555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 113/226] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:02.456160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.491367 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ed4d-cd52-40fd-9e7c-99a72b45cd6c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.517170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.539009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.539861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.540139 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.540407 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.555212 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.557935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:02.569988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:02.571041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:02.571941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 114/227] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:02.572713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.577122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e5bf-3a9e-472b-af50-5acde6e8c935 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.619943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.620708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.621075 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.621450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.622362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=e3eac434204841d49bc8601968cb7f74) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:02.622736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.624655 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.640615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 114/228] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/role_assignments?user.id=e3eac434204841d49bc8601968cb7f74&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:02.641910 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.757892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089f581-cf3d-4b06-9e53-f898b9d945cf None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.827233 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.828723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.829220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.829757 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.832824 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.835146 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.838095 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 115/229] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:02.845284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.852877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b2efc-fd02-46ea-8a32-d99ab7fd2983 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:02.912651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:02.913226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:02.913732 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:02.914114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:02.915096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:02.915479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:02.916946 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:02.921311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.927748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 115/230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:02.933316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:02.956667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:02.975418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5f8d-fdf1-41e2-ae35-c5992fd9e5d4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:03.009690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:03.010557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.010839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.011110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f/users/9cb6c32718a34f96bb2b80c0468dcdc2/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.013121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2721091c11445fc8126d39812016d8f, user_id=9cb6c32718a34f96bb2b80c0468dcdc2, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:03.014722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:03.047900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 116/231] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:02 2022] PUT /identity/v3/projects/e2721091c11445fc8126d39812016d8f/users/9cb6c32718a34f96bb2b80c0468dcdc2/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:03.069603 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c797ac-a627-4a86-9f3a-33b4f33f8a4a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:03.107808 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:03.109127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.109648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.110132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] PATH_INFO: `/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.111664 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] RBAC: Authorizing `identity:get_user(user_id=9cb6c32718a34f96bb2b80c0468dcdc2)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:03.113427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:03.115787 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 116/232] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:02 2022] GET /identity/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:03.121407 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:03.281805 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991320a-7426-452b-ae5a-f787e89a6432 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:03.299199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:03.329608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0d8d3-1db9-4906-8f66-8a7ba3a9ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:03.330900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0d8d3-1db9-4906-8f66-8a7ba3a9ab33 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.331403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0d8d3-1db9-4906-8f66-8a7ba3a9ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.331886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0d8d3-1db9-4906-8f66-8a7ba3a9ab33 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.333405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0d8d3-1db9-4906-8f66-8a7ba3a9ab33 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:03.335081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0d8d3-1db9-4906-8f66-8a7ba3a9ab33 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:03.337044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 117/233] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:03 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:03.343607 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:03.450165 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:03.463611 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3c9a8-9c5e-48a8-9c8a-3aebd4b37084 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:03.499363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeab6d9-b268-4bae-bdd1-7ee64389df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:03.500314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab6d9-b268-4bae-bdd1-7ee64389df57 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.500595 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab6d9-b268-4bae-bdd1-7ee64389df57 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.500868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab6d9-b268-4bae-bdd1-7ee64389df57 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f/users/9cb6c32718a34f96bb2b80c0468dcdc2/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.503191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab6d9-b268-4bae-bdd1-7ee64389df57 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2721091c11445fc8126d39812016d8f, user_id=9cb6c32718a34f96bb2b80c0468dcdc2, role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:03.504672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab6d9-b268-4bae-bdd1-7ee64389df57 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:03.520301 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 117/234] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:03 2022] PUT /identity/v3/projects/e2721091c11445fc8126d39812016d8f/users/9cb6c32718a34f96bb2b80c0468dcdc2/roles/75add8e0c0454b3a8681eab45a26244c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:03.748548 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a900bf-ef84-480b-b2ee-5bda5359ba7e None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.748835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a900bf-ef84-480b-b2ee-5bda5359ba7e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.749115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a900bf-ef84-480b-b2ee-5bda5359ba7e None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.750887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 118/235] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:03.756438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.756729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.757007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.782340 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:03.783561 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:03.866989 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cdfc27-3a0e-4739-8075-8bae5f50d28f None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.867319 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cdfc27-3a0e-4739-8075-8bae5f50d28f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.867545 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cdfc27-3a0e-4739-8075-8bae5f50d28f None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.868976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 119/236] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:03.875688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:03.876065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 118/237] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:03.876181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:03.876263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:03.902574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:03.906457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:03.907664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:03.992182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 120/238] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:03.996604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:04.018069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:04.029783 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9d2c-0cae-4e18-aafb-e541bf141161 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.068193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.069087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.069448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.069721 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.081765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:04.091058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:04.092135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:04.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 119/239] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:04.102210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:04.216282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd466-ea52-42db-b1f5-03ee4aa50230 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.216490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9279aac-c359-49b2-931c-8113a3fd2548 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.229599 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:04.246547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:04.258059 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830707a-c970-437e-842d-b109ee74245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.258752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830707a-c970-437e-842d-b109ee74245b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.259034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830707a-c970-437e-842d-b109ee74245b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.259306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830707a-c970-437e-842d-b109ee74245b None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.260075 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830707a-c970-437e-842d-b109ee74245b None admin] RBAC: Adding query filter params (name=reader) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:04.260378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830707a-c970-437e-842d-b109ee74245b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:04.263018 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830707a-c970-437e-842d-b109ee74245b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:04.279042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 120/240] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:04.284700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830707a-c970-437e-842d-b109ee74245b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:04.288601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.289450 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.289732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.290000 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.294295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:04.296272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:04.298236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 121/241] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:04.303745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:04.403245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830707a-c970-437e-842d-b109ee74245b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.438379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.439282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.439560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.439936 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] PATH_INFO: `/v3/users/82cf86898fc44c029465850a0dcc2f39` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.454148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] RBAC: Authorizing `identity:get_user(user_id=82cf86898fc44c029465850a0dcc2f39)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:04.455626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:04.457373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 121/242] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:04 2022] GET /identity/v3/users/82cf86898fc44c029465850a0dcc2f39 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:04.462791 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:04.514348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:04.549883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a1973-c2c2-42e9-86ff-e5f6edf088d5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.566439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab1f4a-363b-4b7f-a0d9-b5791901fb65 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.601093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.601805 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.602089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.602363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.633959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.635030 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.635519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.636001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.636946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:04.638653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:04.685608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:04.687612 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:04.705495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 122/243] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:04 2022] POST /identity/v3/users => generated 293 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:04.762739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:04.821370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 122/244] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:48:04.822837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:04.862758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:04.949252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90443e0f-6bab-4027-af74-41f436acfd8e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:04.987866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17419339-9cc0-4916-a3d5-1255041988d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:04.988763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17419339-9cc0-4916-a3d5-1255041988d4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:04.989054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17419339-9cc0-4916-a3d5-1255041988d4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:04.989354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17419339-9cc0-4916-a3d5-1255041988d4 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:04.993973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17419339-9cc0-4916-a3d5-1255041988d4 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:04.995457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17419339-9cc0-4916-a3d5-1255041988d4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:04.997590 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 123/245] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:04 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:05.003455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:05.113288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb64908-e302-4cdd-9b69-6e42a34f415e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:05.155500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:05.156175 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:05.156458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:05.156734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:05.157616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] RBAC: Adding query filter params (role.id=e59278c8302b4d63a8f546cccc1ac139, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=82cf86898fc44c029465850a0dcc2f39) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:05.157929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:05.159408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f8df-7425-40f4-b752-9ccd214b6602 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:05.165450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 123/246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:05 2022] GET /identity/v3/role_assignments?user.id=82cf86898fc44c029465850a0dcc2f39&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=e59278c8302b4d63a8f546cccc1ac139 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:05.497641 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b784bd-3cb9-4f60-8648-bc7056b77b8d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:05.497825 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b784bd-3cb9-4f60-8648-bc7056b77b8d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:05.498193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b784bd-3cb9-4f60-8648-bc7056b77b8d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:05.499866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 124/247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:05.505144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:05.505748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:05.506242 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:05.524448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:05.526025 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:05.537911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:05.609196 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 124/248] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:05.635636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b784bd-3cb9-4f60-8648-bc7056b77b8d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:05.751825 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:05.756772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b784bd-3cb9-4f60-8648-bc7056b77b8d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:05.800865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839233-00b3-4fd8-b55f-1f4e27a87b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:05.801727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839233-00b3-4fd8-b55f-1f4e27a87b62 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:05.802011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839233-00b3-4fd8-b55f-1f4e27a87b62 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:05.802288 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839233-00b3-4fd8-b55f-1f4e27a87b62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:05.817446 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44839233-00b3-4fd8-b55f-1f4e27a87b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-44839233-00b3-4fd8-b55f-1f4e27a87b62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:05.827477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:05.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:05.829421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:05.832290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 125/249] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:05.837919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:05.850173 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7e33-4166-44d8-a3a3-19abc45d462d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:05.850458 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7e33-4166-44d8-a3a3-19abc45d462d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:05.850730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7e33-4166-44d8-a3a3-19abc45d462d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:05.852298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 126/250] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:05.858907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:05.859188 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:05.859465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:05.894738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:05.896279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:05.992722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385c72e-ea52-4e87-a030-9c6b9016c229 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.007718 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 127/251] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:06.042362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.052084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.053033 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.053336 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.053606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.054372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:06.054670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:06.056064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:06.072766 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 125/252] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:06.078920 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.121616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:06.209791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:06.225289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eede52d-2dad-48c6-8cfc-8a328d70eac5 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.261211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.261453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fd782-38c7-47be-aa01-01bde6386476 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.261971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.262255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.262534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.273696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:06.281618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:06.282628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:06.283507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:06.289752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 128/253] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:06.295456 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.333072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.334067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.334353 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.334626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] PATH_INFO: `/v3/users/e3eac434204841d49bc8601968cb7f74` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.341456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] RBAC: Authorizing `identity:get_user(user_id=e3eac434204841d49bc8601968cb7f74)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:06.343114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:06.345014 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 126/254] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:06 2022] GET /identity/v3/users/e3eac434204841d49bc8601968cb7f74 => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:06.350148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.433523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:06.459014 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ac69d-afe7-4ec0-ab52-7da444750f9a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.461925 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1dc10-36ab-4485-ae2b-7f3834a7b815 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.498849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.499988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.500385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.500796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.502240 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:06.504046 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:06.506303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 127/255] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:06 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:06.511584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.533974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.534677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.534965 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.535243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.536152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:06.536464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:06.538027 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:06.563350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 129/256] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:06.572105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.593618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:06.739123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9c902-e57b-4cca-b58a-1d157982bcb6 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.750740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f22c5d-8361-4776-86cc-890dcff922a0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:06.805403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab180-8ea1-4283-b440-60fcc7dcc40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.806240 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab180-8ea1-4283-b440-60fcc7dcc40a None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.806518 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab180-8ea1-4283-b440-60fcc7dcc40a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.806787 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab180-8ea1-4283-b440-60fcc7dcc40a None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f/users/e3eac434204841d49bc8601968cb7f74/roles/e7782d74fe1a4e02b57e8e7377ba7e89` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.809514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab180-8ea1-4283-b440-60fcc7dcc40a None admin] RBAC: Authorizing `identity:create_grant(project_id=e2721091c11445fc8126d39812016d8f, user_id=e3eac434204841d49bc8601968cb7f74, role_id=e7782d74fe1a4e02b57e8e7377ba7e89)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:06.811061 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab180-8ea1-4283-b440-60fcc7dcc40a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:06.815679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030751e-2507-4c7b-88ae-a2680dbb1fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:06.816578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030751e-2507-4c7b-88ae-a2680dbb1fe8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.816858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030751e-2507-4c7b-88ae-a2680dbb1fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.817131 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030751e-2507-4c7b-88ae-a2680dbb1fe8 None admin] PATH_INFO: `/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.818974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030751e-2507-4c7b-88ae-a2680dbb1fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=9cb6c32718a34f96bb2b80c0468dcdc2)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:06.820422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030751e-2507-4c7b-88ae-a2680dbb1fe8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:06.822279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 130/257] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:06 2022] GET /identity/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:06.824619 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcd8ab-60ac-47a2-b32a-dc52831f61c9 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.824907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcd8ab-60ac-47a2-b32a-dc52831f61c9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.825183 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcd8ab-60ac-47a2-b32a-dc52831f61c9 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.826954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 131/258] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:06.827974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcd8ab-60ac-47a2-b32a-dc52831f61c9 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:06.840231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 128/259] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:06 2022] PUT /identity/v3/projects/e2721091c11445fc8126d39812016d8f/users/e3eac434204841d49bc8601968cb7f74/roles/e7782d74fe1a4e02b57e8e7377ba7e89 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:06.841724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:06.842065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:06.842350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:06.853384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:06.871278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:06.872520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:07.015806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:07.019671 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 129/260] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:07.053038 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcd8ab-60ac-47a2-b32a-dc52831f61c9 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:07.056950 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:07.154557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:07.155234 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:07.155515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:07.155786 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:07.200057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19a3b-0f32-4a58-91ad-a2c6ef33447d None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:07.234247 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:07.245266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:07.246105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:07.246387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:07.246657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:07.269341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:07.284254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:07.285520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:07.286421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 130/261] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:07.292310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:07.295488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:07.296976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:07.440018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:07.467715 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 132/262] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:07.478201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:07.485400 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096b6cc-a622-4600-a088-a8d26e38bee2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:07.521496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:07.522318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:07.522702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:07.523113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:07.523993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:07.524408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:07.526547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:07.535392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 131/263] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:07.542326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:07.600362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479926b-bb8b-4211-9ffe-12ee989d79f9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:07.636347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:07.646398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:07.647210 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:07.647451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:07.647722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:07.648974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:07.650488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:07.657425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 133/264] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:07 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:07.662297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:07.737868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7c691-0be6-4b35-a6a2-cdb8717e700b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:07.765070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:07.814453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:07.815358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:07.815639 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:07.815915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] PATH_INFO: `/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:07.821321 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] RBAC: Authorizing `identity:get_user(user_id=9cb6c32718a34f96bb2b80c0468dcdc2)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:07.822788 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:07.824491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 132/265] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:07 2022] GET /identity/v3/users/9cb6c32718a34f96bb2b80c0468dcdc2 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:07.830711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:07.870328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e797d1-2e68-4e3a-acc8-69fa5453b2b7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:07.952597 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:07.953354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:07.953635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:07.953907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:07.954749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=9cb6c32718a34f96bb2b80c0468dcdc2) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:07.955104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:07.956573 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:07.968557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 134/266] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:07 2022] GET /identity/v3/role_assignments?user.id=9cb6c32718a34f96bb2b80c0468dcdc2&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:07.985717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941baa3b-2f06-4d2a-98d1-342e55d6c94d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:08.011608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:08.027386 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:08.028467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.028986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.029576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.112119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:08.114468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:08.230256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 133/267] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:08.238496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:08.375255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02531-8081-4b91-9498-d4288d0b1e70 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:08.385963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:08.418489 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80263b-765c-4eb4-b3b8-048e06b9461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:08.419486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80263b-765c-4eb4-b3b8-048e06b9461f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.419980 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80263b-765c-4eb4-b3b8-048e06b9461f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.420585 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80263b-765c-4eb4-b3b8-048e06b9461f None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.422398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80263b-765c-4eb4-b3b8-048e06b9461f None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:08.423857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80263b-765c-4eb4-b3b8-048e06b9461f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:08.425622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 135/268] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:08 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:08.431985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:08.464693 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:08.483598 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e47e7-1533-4a79-a863-07075843522c None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.483964 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e47e7-1533-4a79-a863-07075843522c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.484142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e47e7-1533-4a79-a863-07075843522c None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.485761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 136/269] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:08.491193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.491507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.491744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.530633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:08.531953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:08.591076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c436-23b8-452a-9d9d-5a67fe9bbf40 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:08.635856 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 137/270] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:08.651273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:08.651953 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.652235 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.652512 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.653398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] RBAC: Adding query filter params (role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=9cb6c32718a34f96bb2b80c0468dcdc2) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:08.653712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:08.655223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adea2be-ddc0-40a5-a4fd-7f78efde8e8b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:08.661923 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 134/271] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:08 2022] GET /identity/v3/role_assignments?user.id=9cb6c32718a34f96bb2b80c0468dcdc2&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=75add8e0c0454b3a8681eab45a26244c => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:08.674680 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:08.750686 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20851ec7-7e3b-4db0-b2cd-e97660496b2a None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.751033 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20851ec7-7e3b-4db0-b2cd-e97660496b2a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.751212 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20851ec7-7e3b-4db0-b2cd-e97660496b2a None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.752986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 135/272] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:08.767221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.767562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.767762 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.791500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:08.792744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:08.838354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25ec4e-21fa-4e70-b443-3a017179c9d3 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:08.850263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:08.882607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bae0ba-7779-443d-bb76-ab108523a047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:08.882607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bae0ba-7779-443d-bb76-ab108523a047 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.882926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bae0ba-7779-443d-bb76-ab108523a047 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.883004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bae0ba-7779-443d-bb76-ab108523a047 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.884423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bae0ba-7779-443d-bb76-ab108523a047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:08.885718 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bae0ba-7779-443d-bb76-ab108523a047 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:08.887882 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 138/273] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:08.889185 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 136/274] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:08.889352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555958e6-5681-479d-87f7-eb75918706ca None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.889638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555958e6-5681-479d-87f7-eb75918706ca None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.889916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555958e6-5681-479d-87f7-eb75918706ca None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.891396 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 139/275] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:08.895853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:08.896730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:08.897018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:08.899096 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:08.920865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:08.921068 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:08.921789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:08.998679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 140/276] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:09.000010 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.024667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578943e-d862-4ae1-ba83-bc809b089aec None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.064945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.065656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.065943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.066218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.066934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.068407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.090687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 137/277] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:08 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:09.098362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.109457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9b6eb-4119-4733-bbe5-cba29b7a0b95 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.119700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:09.165558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.166881 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.167336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.168324 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.185006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:09.192170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:09.197943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:09.199203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:09.200282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:09.204254 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 141/278] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:09.210483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.238983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d736-eaf6-4d97-8b9c-f0872e88d0fd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.292385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.293796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.294229 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.295576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.309545 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:09.319406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:09.320625 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:09.321919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:09.324177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 138/279] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:09.330097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.341793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976a9a9-0432-4d06-9086-cd925d883b03 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.388174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.389205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.389894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.390343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.391352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:09.391815 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.393400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.404852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 142/280] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:09.411175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.447531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:09.458986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f4203-05ee-4a99-acf1-f8c929b0d636 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.501681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:09.507521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.508207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.508489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.508764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.509579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:09.509890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.511305 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.526564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 139/281] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:09.527890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.546504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637825-8a0a-4aba-aefa-641e54bde926 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.590716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.591626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.591953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.592231 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] PATH_INFO: `/v3/users/82cf86898fc44c029465850a0dcc2f39` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.593990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] RBAC: Authorizing `identity:get_user(user_id=82cf86898fc44c029465850a0dcc2f39)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.596198 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.598988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 143/282] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/users/82cf86898fc44c029465850a0dcc2f39 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:09.604830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.652575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c90c-fd29-4c80-bdf0-43e96cd02f32 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.687591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.688516 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.688797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.689070 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] PATH_INFO: `/v3/users/d1d1059d24b54c5eab75ac3ce2d04972` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.704043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] RBAC: Authorizing `identity:get_user(user_id=d1d1059d24b54c5eab75ac3ce2d04972)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.705680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.712297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 140/283] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/users/d1d1059d24b54c5eab75ac3ce2d04972 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:09.713728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.718702 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fed620-826f-42d7-964e-bec463c334f0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.721792 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:09.772962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.774960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.778922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.779417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.781988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.783623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.786118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 144/284] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:09.795928 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:09.817977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:09.907338 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc43da1-77f3-4bc6-b2d6-0c8f733af48c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.969281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162cbc-4d1d-4cce-ba20-90f2f3fc9d00 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:09.972065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:09.972731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:09.973008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:09.973293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:09.974139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, user.id=d1d1059d24b54c5eab75ac3ce2d04972, scope.system=all) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:09.974440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:09.975891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c66f67-8150-4957-90a9-a2f190e57411 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:09.991384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 141/285] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:48:09 2022] GET /identity/v3/role_assignments?user.id=d1d1059d24b54c5eab75ac3ce2d04972&scope.system=all&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:10.020870 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa169f-8a6f-40a3-bd63-db0909078e56 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.021228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa169f-8a6f-40a3-bd63-db0909078e56 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.021458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa169f-8a6f-40a3-bd63-db0909078e56 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:10.023294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 142/286] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:10.028774 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.029091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.029355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:10.036466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:10.037434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.037715 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.037988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f/users/82cf86898fc44c029465850a0dcc2f39/roles/e59278c8302b4d63a8f546cccc1ac139` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:10.040055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2721091c11445fc8126d39812016d8f, user_id=82cf86898fc44c029465850a0dcc2f39, role_id=e59278c8302b4d63a8f546cccc1ac139)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:10.041556 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:10.049496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:10.050709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:10.080676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 145/287] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:09 2022] PUT /identity/v3/projects/e2721091c11445fc8126d39812016d8f/users/82cf86898fc44c029465850a0dcc2f39/roles/e59278c8302b4d63a8f546cccc1ac139 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:10.164654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 143/288] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:10.206810 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:10.250713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:10.336707 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f1e09-87fc-4cee-a429-8307ccf5a808 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:10.373378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:10.374409 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.374559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.374933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:10.386914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:10.395153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:10.396468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:10.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 146/289] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:10.402977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:10.452174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:10.521854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5565677-50dc-4bd0-94b7-b9f3e6f8e3f6 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:10.572741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:10.573465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.573748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.574025 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:10.574802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:10.575109 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:10.576595 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:10.589857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 144/290] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:10.595273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:10.743478 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:10.764373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56171d2-6654-489e-b448-1e25842391f5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:10.798634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:10.799443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.799727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.800004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] PATH_INFO: `/v3/users/e3eac434204841d49bc8601968cb7f74` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:10.801208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] RBAC: Authorizing `identity:get_user(user_id=e3eac434204841d49bc8601968cb7f74)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:10.802692 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:10.804682 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 147/291] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:10 2022] GET /identity/v3/users/e3eac434204841d49bc8601968cb7f74 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:10.810273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:10.925551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbdf97-4700-4367-ab3a-062a583e7fcb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:10.939215 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:10.969096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:10.969791 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:10.970071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:10.970350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:11.043327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:11.044849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:11.147245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 145/292] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:11.152708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:11.320931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6078a2-b084-4bfc-9631-aa1535063a79 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:11.369932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe832ce-a9e0-4e47-b915-f242c30e7314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:11.371181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe832ce-a9e0-4e47-b915-f242c30e7314 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:11.371676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe832ce-a9e0-4e47-b915-f242c30e7314 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:11.372187 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe832ce-a9e0-4e47-b915-f242c30e7314 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:11.373709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe832ce-a9e0-4e47-b915-f242c30e7314 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:11.375832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe832ce-a9e0-4e47-b915-f242c30e7314 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:11.378128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 148/293] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:11 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:11.386356 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:11.397445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:11.533083 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472490-7fd4-4563-b3da-82c6e9d26fbd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:11.567636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:11.568296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:11.568574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:11.568846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:11.569718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=e3eac434204841d49bc8601968cb7f74) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:11.570028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:11.571471 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65c140-3d42-4d89-9225-c3145c7faeff None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:11.578106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 146/294] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:11 2022] GET /identity/v3/role_assignments?user.id=e3eac434204841d49bc8601968cb7f74&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:11.859398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99691d0-e270-459b-a2db-b4dcf9a8eb44 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:11.859690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99691d0-e270-459b-a2db-b4dcf9a8eb44 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:11.859973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99691d0-e270-459b-a2db-b4dcf9a8eb44 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:11.861725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 149/295] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:11.867291 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:11.867579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:11.867857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:11.899570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:11.899570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:11.974789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 147/296] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:11.999379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99691d0-e270-459b-a2db-b4dcf9a8eb44 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:12.050275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:12.190841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99691d0-e270-459b-a2db-b4dcf9a8eb44 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:12.254544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47544be3-5bbf-4ccd-996b-71e3b354efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:12.255370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47544be3-5bbf-4ccd-996b-71e3b354efdd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.255653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47544be3-5bbf-4ccd-996b-71e3b354efdd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.255925 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47544be3-5bbf-4ccd-996b-71e3b354efdd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.271003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47544be3-5bbf-4ccd-996b-71e3b354efdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-47544be3-5bbf-4ccd-996b-71e3b354efdd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:12.281769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:12.283129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:12.284119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:12.292440 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 150/297] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:12.295752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:12.299969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:12.397831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1efda-1674-4fec-8a43-cce8d00dd1c8 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.398382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1efda-1674-4fec-8a43-cce8d00dd1c8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.398382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1efda-1674-4fec-8a43-cce8d00dd1c8 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.399640 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 151/298] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:12.405816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.406113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.406395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.453681 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:12.455121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:12.463112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64905dea-f90e-4090-a994-b82d66021d55 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:12.502076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:12.502784 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.503192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.503629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.504627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:12.505197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:12.506752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:12.521600 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 148/299] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:12.523218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:12.537367 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 152/300] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:12.553750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:12.565886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:12.592189 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:12.636764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1896aa-da62-4841-86c5-6cb366a01123 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:12.671958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036a65e-f586-45fa-b744-2cd2ff5fecc3 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:12.675215 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:12.676375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.676859 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.677405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] PATH_INFO: `/v3/users/173f3585a5824814b94f06f3c3cec061` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.692229 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] RBAC: Authorizing `identity:get_user(user_id=173f3585a5824814b94f06f3c3cec061)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:12.694259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:12.701615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 149/301] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:12 2022] GET /identity/v3/users/173f3585a5824814b94f06f3c3cec061 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:12.703143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:12.707523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:12.708650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.709136 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.709650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.711074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:12.712791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:12.721441 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 153/302] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:12.725538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:12.811180 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:12.819965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1736-e56c-4a18-96b0-652f6cc8287c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:12.835004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c80aeb-1f64-47f4-9996-fce2493b1dde None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:12.848471 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:12.862047 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:12.862722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.863008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.863286 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.864122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] RBAC: Adding query filter params (role.id=e59278c8302b4d63a8f546cccc1ac139, user.id=173f3585a5824814b94f06f3c3cec061, scope.system=all) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:12.864605 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:12.866110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4923227-0af8-4141-84e7-66776a56cdeb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:12.871079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a057bd7-057c-4971-8a43-1a4554db00b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:12.871588 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 150/303] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:48:12 2022] GET /identity/v3/role_assignments?user.id=173f3585a5824814b94f06f3c3cec061&scope.system=all&role.id=e59278c8302b4d63a8f546cccc1ac139 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:12.872716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fbad-9d53-487f-8843-59b81e286096 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.873001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fbad-9d53-487f-8843-59b81e286096 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.873390 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fbad-9d53-487f-8843-59b81e286096 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.873889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057bd7-057c-4971-8a43-1a4554db00b8 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.874279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057bd7-057c-4971-8a43-1a4554db00b8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.874654 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057bd7-057c-4971-8a43-1a4554db00b8 None admin] PATH_INFO: `/v3/groups` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.875523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a057bd7-057c-4971-8a43-1a4554db00b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:12.877272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a057bd7-057c-4971-8a43-1a4554db00b8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:12.879875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 151/304] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:12.881560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.882053 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.882496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.905431 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 154/305] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 14:48:12 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:48:12.906550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:12.908141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:12.920448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a7b52-9be6-407f-be28-bb52d0557970 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.920731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a7b52-9be6-407f-be28-bb52d0557970 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.921019 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a7b52-9be6-407f-be28-bb52d0557970 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.923661 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 155/306] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:12.931587 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:12.931860 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:12.932007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:12.952243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:12.953609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:13.001675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 152/307] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:13.021709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.037304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 156/308] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:13.060527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.140271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a33e-aed0-4eca-90fc-acb162832d9d None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.147741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:13.162077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:13.188204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.188750 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.189034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.189345 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.191322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbdac0-e0bb-454e-a627-5eb5335a7204 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.207247 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:13.217058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:13.218100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 153/309] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:13.225041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.239941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.241714 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.242212 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.243120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.258789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:13.270164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:13.271488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:13.272483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:13.275803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 157/310] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:13.281703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.359834 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e928-1f02-4e7e-96e2-40b031194d4a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.386478 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54134a5c-270f-4465-86b8-719b68304531 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.403984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.404909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.407116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.407704 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.408717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:13.409224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:13.411680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:13.421619 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 154/311] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:13.421827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:13.426928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.434999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.435699 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.435984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.436261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.437088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:13.437577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:13.439594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:13.457390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 158/312] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:13.463967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.480220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:13.547204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2071a8-6ee0-409a-bfa6-2ba7a009462d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.584849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.585814 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.586088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.586354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] PATH_INFO: `/v3/users/d1d1059d24b54c5eab75ac3ce2d04972` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.587568 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d1d1059d24b54c5eab75ac3ce2d04972)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:13.589138 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:13.595405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 155/313] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/users/d1d1059d24b54c5eab75ac3ce2d04972 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:13.597703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.603321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798ad15-646c-49be-9f8f-8ed22c535377 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.638316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.639335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.639569 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.639838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] PATH_INFO: `/v3/users/82cf86898fc44c029465850a0dcc2f39` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.641042 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] RBAC: Authorizing `identity:get_user(user_id=82cf86898fc44c029465850a0dcc2f39)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:13.642560 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:13.648953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 159/314] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/users/82cf86898fc44c029465850a0dcc2f39 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:13.651919 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:13.722118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b213f-ebab-41b5-8e28-0a1d9593c8b7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.729198 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:13.731228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:13.763431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.764206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.764485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.764754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] PATH_INFO: `/v3/system/users/d1d1059d24b54c5eab75ac3ce2d04972/roles/e7782d74fe1a4e02b57e8e7377ba7e89` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.766320 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d1d1059d24b54c5eab75ac3ce2d04972, role_id=e7782d74fe1a4e02b57e8e7377ba7e89)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:13.767796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:13.786062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 156/315] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 14:48:13 2022] PUT /identity/v3/system/users/d1d1059d24b54c5eab75ac3ce2d04972/roles/e7782d74fe1a4e02b57e8e7377ba7e89 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:13.790686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa23d7-26f1-4368-9640-e200c0af10bd None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:13.833211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:13.833819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:13.834097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:13.834369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:13.908808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:13.910440 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:13.971893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:14.026900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 160/316] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:48:14.032922 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:14.140833 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea8804-1b22-4cd9-a603-79200edc56b9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:14.177825 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742b16b-997f-4017-a0ae-2dbf6f9e0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:14.178706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742b16b-997f-4017-a0ae-2dbf6f9e0927 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:14.178980 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742b16b-997f-4017-a0ae-2dbf6f9e0927 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:14.179246 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742b16b-997f-4017-a0ae-2dbf6f9e0927 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:14.180439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742b16b-997f-4017-a0ae-2dbf6f9e0927 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:14.181939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742b16b-997f-4017-a0ae-2dbf6f9e0927 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:14.184080 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 157/317] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:14 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:14.189954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:14.302067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b04993-309e-4c40-9042-4647e4d32cda None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:14.336827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:14.337519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:14.337800 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:14.338071 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:14.338898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] RBAC: Adding query filter params (role.id=e59278c8302b4d63a8f546cccc1ac139, scope.project.id=e2721091c11445fc8126d39812016d8f, user.id=82cf86898fc44c029465850a0dcc2f39) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:14.339206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:14.340653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b1f1c-e5ef-4032-bd76-092ab89e5c96 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:14.347096 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 161/318] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:14 2022] GET /identity/v3/role_assignments?user.id=82cf86898fc44c029465850a0dcc2f39&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=e59278c8302b4d63a8f546cccc1ac139 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:15.137049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe1de-839a-4cac-bcb1-e79c51c7a35c None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.137654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe1de-839a-4cac-bcb1-e79c51c7a35c None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.137654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe1de-839a-4cac-bcb1-e79c51c7a35c None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.139037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 158/319] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:15.144783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8b557-d033-42d2-979d-0b54a3673b69 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.145266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8b557-d033-42d2-979d-0b54a3673b69 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.145356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8b557-d033-42d2-979d-0b54a3673b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.163548 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-07d8b557-d033-42d2-979d-0b54a3673b69 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:15.164770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-07d8b557-d033-42d2-979d-0b54a3673b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:15.182502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:15.237282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 162/320] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:15.259393 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fe1de-839a-4cac-bcb1-e79c51c7a35c None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:15.285466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e957aa-9cc5-40cf-ac8f-78c0834752aa None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.285807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e957aa-9cc5-40cf-ac8f-78c0834752aa None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.286217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e957aa-9cc5-40cf-ac8f-78c0834752aa None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.287742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 163/321] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:15.292669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.292958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.293265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.307644 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:15.310232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:15.311571 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:15.376622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fe1de-839a-4cac-bcb1-e79c51c7a35c None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:15.389360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 164/322] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:15.411514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:15.412299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:15.412419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.412586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.412884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.423844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:15.431836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:15.432812 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:15.434125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 159/323] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:15.438732 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:15.506661 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:15.528144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc19f8-a14e-41b0-8228-d7dd2d8b3d15 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:15.538646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:15.544793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f203a-e771-465e-8690-72e9c7fa8187 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:15.573081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:15.573932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.574193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.574489 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.575824 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:15.577403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:15.579238 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 165/324] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:15.584712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:15.591533 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:15.592286 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.592556 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.592836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.593613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:15.593937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:15.595321 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:15.604201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 160/325] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:15.610030 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:15.704181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a202a-dca6-4ab6-97ee-0b9cab78812d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:15.740856 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:15.741561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.741675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045cdec-21d0-405c-8f6d-6b5d2472c9f8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:15.741888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.742112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] PATH_INFO: `/v3/groups` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.742810 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:15.744613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:15.759739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 166/326] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 14:48:15 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:48:15.783529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6145486-776b-4787-8c06-3f4741c44c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:15.784565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6145486-776b-4787-8c06-3f4741c44c59 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.784923 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6145486-776b-4787-8c06-3f4741c44c59 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.785345 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6145486-776b-4787-8c06-3f4741c44c59 None admin] PATH_INFO: `/v3/users/173f3585a5824814b94f06f3c3cec061` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.786856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6145486-776b-4787-8c06-3f4741c44c59 None admin] RBAC: Authorizing `identity:get_user(user_id=173f3585a5824814b94f06f3c3cec061)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:15.788364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6145486-776b-4787-8c06-3f4741c44c59 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:15.790348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 161/327] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:15 2022] GET /identity/v3/users/173f3585a5824814b94f06f3c3cec061 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:15.796118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:15.810902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:15.914429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b45dd-00d0-4b4f-849d-f4ca2cb82e02 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:15.953232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27585ca-20be-4750-a376-a75866b38577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:15.954551 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27585ca-20be-4750-a376-a75866b38577 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:15.954551 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27585ca-20be-4750-a376-a75866b38577 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:15.954743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27585ca-20be-4750-a376-a75866b38577 None admin] PATH_INFO: `/v3/system/users/173f3585a5824814b94f06f3c3cec061/roles/e59278c8302b4d63a8f546cccc1ac139` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:15.957185 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27585ca-20be-4750-a376-a75866b38577 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=173f3585a5824814b94f06f3c3cec061, role_id=e59278c8302b4d63a8f546cccc1ac139)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:15.958239 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27585ca-20be-4750-a376-a75866b38577 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:15.969388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 167/328] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 14:48:15 2022] PUT /identity/v3/system/users/173f3585a5824814b94f06f3c3cec061/roles/e59278c8302b4d63a8f546cccc1ac139 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:16.102171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1bdb0-930b-46a8-aa3e-131a4ca06276 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:16.102511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1bdb0-930b-46a8-aa3e-131a4ca06276 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:16.102736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1bdb0-930b-46a8-aa3e-131a4ca06276 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:16.104318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 162/329] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:16.110496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:16.111156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:16.111641 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:16.130451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:16.132097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:16.206991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 168/330] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:16.230815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1bdb0-930b-46a8-aa3e-131a4ca06276 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:16.264231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:16.349662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1bdb0-930b-46a8-aa3e-131a4ca06276 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:16.387340 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:16.388186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:16.388469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:16.388874 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:16.401261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:16.409440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:16.410432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:16.411317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 163/331] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:16.416800 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:16.426266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:16.531034 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90004469-12ee-4679-a9cd-c200a96e57ec None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:16.569980 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:16.570680 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:16.570968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:16.571240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:16.572005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] RBAC: Adding query filter params (name=member) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:16.572314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:16.573764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:16.583175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 169/332] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:16.588701 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:16.694314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:16.708290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f28880-27be-4df9-9347-026819aa58ce None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:16.749621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fdb0-cdc4-4f96-8eb6-4455a9182204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:16.750537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fdb0-cdc4-4f96-8eb6-4455a9182204 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:16.750819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fdb0-cdc4-4f96-8eb6-4455a9182204 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:16.751094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fdb0-cdc4-4f96-8eb6-4455a9182204 None admin] PATH_INFO: `/v3/users/d1d1059d24b54c5eab75ac3ce2d04972` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:16.752348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fdb0-cdc4-4f96-8eb6-4455a9182204 None admin] RBAC: Authorizing `identity:get_user(user_id=d1d1059d24b54c5eab75ac3ce2d04972)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:16.753871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fdb0-cdc4-4f96-8eb6-4455a9182204 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:16.755707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 164/333] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:16 2022] GET /identity/v3/users/d1d1059d24b54c5eab75ac3ce2d04972 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:16.761648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:16.835517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:16.879825 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af933930-0d42-43c7-9063-dcae13610ea8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:16.915834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:16.916513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:16.916798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:16.917078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:16.917952 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] RBAC: Adding query filter params (role.id=e7782d74fe1a4e02b57e8e7377ba7e89, user.id=d1d1059d24b54c5eab75ac3ce2d04972, scope.system=all) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:16.918266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:16.919728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953a264-a31a-4b9f-b887-2600f441b28a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:16.934728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 170/334] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:48:16 2022] GET /identity/v3/role_assignments?user.id=d1d1059d24b54c5eab75ac3ce2d04972&scope.system=all&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:18.430583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c7f5-362e-4bdf-9303-c13f01db1be6 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:18.431126 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c7f5-362e-4bdf-9303-c13f01db1be6 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:18.431126 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c7f5-362e-4bdf-9303-c13f01db1be6 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:18.432805 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 165/335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:18.438504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b54209-c7d4-4224-acbe-0831e3c7f29d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:18.438956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b54209-c7d4-4224-acbe-0831e3c7f29d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:18.439131 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b54209-c7d4-4224-acbe-0831e3c7f29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:18.462567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a2b54209-c7d4-4224-acbe-0831e3c7f29d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:18.463772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a2b54209-c7d4-4224-acbe-0831e3c7f29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:18.553093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 171/336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:18.592646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c7f5-362e-4bdf-9303-c13f01db1be6 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:18.678932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5be57-3a4a-4947-a948-ed26cf5ad15c None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:18.679604 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5be57-3a4a-4947-a948-ed26cf5ad15c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:18.680161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5be57-3a4a-4947-a948-ed26cf5ad15c None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:18.682160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 172/337] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:18.688567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:18.689146 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:18.689697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:18.730398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:18.731991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:18.751273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c7f5-362e-4bdf-9303-c13f01db1be6 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:18.804717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:18.805860 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:18.806305 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:18.806740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:18.830891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:18.835138 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:18.843461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:18.844756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:18.850034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:18.852602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 166/338] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:18.864091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:18.864519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 173/339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:18.890752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:18.902180 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.004408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c057438-e8ed-4cca-b3af-970a68795a4a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.047373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.048069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.048351 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.048627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.049425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:19.049734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:19.051468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:19.072912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 167/340] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:19.080723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92dfa-8397-4670-b59d-43351be9c1b8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.086270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.152650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.153807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.154260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.154698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.176684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:19.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:19.195050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 14:48:19.196153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:19.199008 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 174/341] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:19.208063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.242053 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:19.281581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587066d-9fdb-4a88-ad15-cb48ac41f325 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.304881 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:19.348453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c26205-87f5-4118-aeb9-155956904e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.349412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c26205-87f5-4118-aeb9-155956904e8f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.349696 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c26205-87f5-4118-aeb9-155956904e8f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.349955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c26205-87f5-4118-aeb9-155956904e8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.411465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b55f-6d8b-4084-8376-40359ef9b9bf None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.434705 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c26205-87f5-4118-aeb9-155956904e8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:19.436847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c26205-87f5-4118-aeb9-155956904e8f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:19.463398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.464069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.464351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.464620 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.465404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] RBAC: Adding query filter params (name=reader) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:19.465709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:19.467303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362fee7-58ae-48f1-98d6-3eaa331f9712 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:19.481184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 175/342] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:19.482671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325fa26-2359-4d08-b912-31fb359f1a48 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.482955 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325fa26-2359-4d08-b912-31fb359f1a48 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.483230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325fa26-2359-4d08-b912-31fb359f1a48 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.489558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 176/343] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:19.490342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325fa26-2359-4d08-b912-31fb359f1a48 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.527966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:19.553181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 168/344] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:48:19.555489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.555941 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.556379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.583293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:19.584772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:19.628430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325fa26-2359-4d08-b912-31fb359f1a48 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.638676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:19.672562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.674522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.674809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.675080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] PATH_INFO: `/v3/users/173f3585a5824814b94f06f3c3cec061` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.677194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=173f3585a5824814b94f06f3c3cec061)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:19.680453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:19.683909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 177/345] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/users/173f3585a5824814b94f06f3c3cec061 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:19.684756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.718490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 169/346] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:19.719737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.813346 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025a596-3580-4b5f-bdc7-717285154bc5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.850690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2530765-494c-4a9f-a0ec-ced6298b633b None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:19.851582 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839c143-0223-44b9-a1a1-11802f9bba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839c143-0223-44b9-a1a1-11802f9bba95 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.852746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839c143-0223-44b9-a1a1-11802f9bba95 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.853012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839c143-0223-44b9-a1a1-11802f9bba95 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.855198 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839c143-0223-44b9-a1a1-11802f9bba95 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:19.855916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839c143-0223-44b9-a1a1-11802f9bba95 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:19.858072 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 178/347] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:19.859444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff07d8-07e6-4ad0-a68a-02b5a7220042 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.859832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff07d8-07e6-4ad0-a68a-02b5a7220042 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.860211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff07d8-07e6-4ad0-a68a-02b5a7220042 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.861856 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 179/348] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:19.863011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff07d8-07e6-4ad0-a68a-02b5a7220042 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.865177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:19.902307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:19.902987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:19.903268 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:19.903541 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:19.904440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] RBAC: Adding query filter params (role.id=e59278c8302b4d63a8f546cccc1ac139, user.id=173f3585a5824814b94f06f3c3cec061, scope.system=all) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:19.904751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:19.906315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:19.919624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 170/349] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/role_assignments?user.id=173f3585a5824814b94f06f3c3cec061&scope.system=all&role.id=e59278c8302b4d63a8f546cccc1ac139 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:19.920354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:19.920582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:19.996065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff07d8-07e6-4ad0-a68a-02b5a7220042 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.034328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba074-aede-4e09-94d3-212a37d23f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.035696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba074-aede-4e09-94d3-212a37d23f88 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.036199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba074-aede-4e09-94d3-212a37d23f88 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.036731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba074-aede-4e09-94d3-212a37d23f88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.050398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba074-aede-4e09-94d3-212a37d23f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-1fbba074-aede-4e09-94d3-212a37d23f88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:20.064102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:20.065123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:20.066063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:20.070034 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 180/350] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:20.071610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.072113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.072600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.091908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:20.093554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:20.107704 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f593d-0bd4-4785-8bc4-82c29e8a46d7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.169835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.169835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.169835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.169835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.181181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:20.181515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 181/351] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:20.182408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.183582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:20.186996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 171/352] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:19 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:20.194158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.215338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:20.260929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:20.299545 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae1186-d8c1-46c1-a29f-e9ddc9af02bf None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.340847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901d376-9a6a-4352-aa8d-60e2f8985a4e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.357405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.358085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.358374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.358696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.359505 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:20.359821 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:20.364199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:20.373819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 182/353] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:20.374781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.380459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.381191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.381501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.381776 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.382680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=6f218d2e4a774028b7261264ea3fdd59) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:20.382988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:20.384508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:20.392269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 172/354] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:20.393445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.504932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:20.517230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0c0f5-f93e-4eea-a779-eaf9d76e3aae None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.532120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:20.542141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef02d3-9ec0-4795-ba23-696ec5ff55a4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.574100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.574955 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.575241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.575517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.593617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:20.594419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd43cd-9a83-4246-9cb1-9e0511bd53e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.595098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd43cd-9a83-4246-9cb1-9e0511bd53e7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.595378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd43cd-9a83-4246-9cb1-9e0511bd53e7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.595651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd43cd-9a83-4246-9cb1-9e0511bd53e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:20.601962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:20.603035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:20.605764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 183/355] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:20.610993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.690804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd43cd-9a83-4246-9cb1-9e0511bd53e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:20.692784 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd43cd-9a83-4246-9cb1-9e0511bd53e7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:20.737138 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a412b-a436-48e5-a840-0c587194d1d5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.779286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.779984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.780266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.780542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.781341 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:20.781651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:20.783045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:20.792854 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 184/356] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:20.798938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.821139 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:20.824585 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:20.837392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 173/357] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:20.838883 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33dafb-6fec-4d32-bf7b-1e0d389be10e None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.839171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33dafb-6fec-4d32-bf7b-1e0d389be10e None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.839450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33dafb-6fec-4d32-bf7b-1e0d389be10e None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.841501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 174/358] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:20.845415 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33dafb-6fec-4d32-bf7b-1e0d389be10e None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:20.926243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdbb9e-1064-4478-82dc-069bfffb57a5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:20.963327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:20.963992 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:20.964283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:20.964546 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:20.985998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33dafb-6fec-4d32-bf7b-1e0d389be10e None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.027782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9d2aa-86ef-41d6-ad37-b104491304f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.028665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d2aa-86ef-41d6-ad37-b104491304f7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.028945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d2aa-86ef-41d6-ad37-b104491304f7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.031535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9d2aa-86ef-41d6-ad37-b104491304f7 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.032845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9d2aa-86ef-41d6-ad37-b104491304f7 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.034408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9d2aa-86ef-41d6-ad37-b104491304f7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.036969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 175/359] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:21.038414 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.038698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.038975 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.050333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.052370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.057838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:21.059125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:21.074185 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:21.132088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:21.145844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 176/360] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:21.146928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.179727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 185/361] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:21.186060 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.292446 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7647832-b371-4e23-9ae7-81b6c24484a2 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.305143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e34e9a-9608-470f-b4d3-5355f192700d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.327214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:21.342222 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.343682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.343962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.344287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.345502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.347064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.347907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b97d-db3d-4534-9ae2-465b84a20eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.348861 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 177/362] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:21.348981 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b97d-db3d-4534-9ae2-465b84a20eb9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.349172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b97d-db3d-4534-9ae2-465b84a20eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.349448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b97d-db3d-4534-9ae2-465b84a20eb9 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.349568 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.350633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b97d-db3d-4534-9ae2-465b84a20eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.352124 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b97d-db3d-4534-9ae2-465b84a20eb9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.354146 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 186/363] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:21.357491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e81df-b51f-4e5d-9923-02997f8f2cea None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.357783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e81df-b51f-4e5d-9923-02997f8f2cea None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.358095 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e81df-b51f-4e5d-9923-02997f8f2cea None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.359389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 187/364] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:21.365089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.365402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.365672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.386763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:21.387963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:21.420517 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:21.469529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 188/365] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:21.470468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.475786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413125a-33cd-431f-880b-25763714dd70 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.522168 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.523006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.523281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.523547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.538703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:21.547141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:21.548206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 178/366] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:21.550536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.607536 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b23e6-13bd-4c64-b459-477a233b9bd2 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.625622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:21.663852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.664732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.665005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.665327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.666520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.667959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.671018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 189/367] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:21.672091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.692504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ddc2-d5f9-4db5-877f-efa9418b4026 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.715560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:21.732457 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.733125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.733440 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.733710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.734551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=6f218d2e4a774028b7261264ea3fdd59) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:21.734856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.736289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.742695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 179/368] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=75add8e0c0454b3a8681eab45a26244c => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:21.743630 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.826904 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4fd5a-8cbf-46f0-8e98-f542ae01299f None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.873697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.874533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.874819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.875086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.889889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:21.891936 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33660c-7f77-4871-95d3-a954f31c3f00 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:21.898782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:21.899835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:21.900809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 190/369] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:21.901744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.933929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:21.934626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:21.935615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:21.935900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:21.936682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:21.936991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:21.938824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:21.950926 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 180/370] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:21.951848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:21.984816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.029875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.052841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c5df7-34d4-4f2c-99eb-9c4f2669a3ef None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.096262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.097039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.097417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.097760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.098658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a831e-8b58-4eec-9e83-11e180bda3ca None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.098869 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=e2721091c11445fc8126d39812016d8f) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:22.099040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.101019 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.108770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 191/371] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:22.109756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:22.142592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.144314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.144815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.145299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.147154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:22.147931 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.149582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.162069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 181/372] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:22.168722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:22.228098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c08286-a25f-4594-bd17-43fdd98270a2 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.241682 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.268541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.269447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.269922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.270265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.279058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3c903-92f6-47e7-89e9-905977e48ab9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.288896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.318251 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.318796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.319078 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.319427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.350623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.352463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.401496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.403349 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.476304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 192/373] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:22.482732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:22.499508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.622368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161aa1-32df-460b-b473-220dbb0b7f04 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.623210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 182/374] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:22.629873 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:22.657708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.680716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.680716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.680716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.680716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.686082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.687570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.689817 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 193/375] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:22.695703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:22.804185 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1a654-2b28-4457-9e21-bde52d06de1d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.833627 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:22.874069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.876049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.877103 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.877617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] PATH_INFO: `/v3/projects/5967761725a6499fb0767a5715d5e41f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.880365 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5b689-44af-406d-81f9-346ef1fee60d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:22.881327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] RBAC: Authorizing `identity:get_project(project_id=5967761725a6499fb0767a5715d5e41f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.882993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.887577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 183/376] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/projects/5967761725a6499fb0767a5715d5e41f => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:22.893940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:22.939753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:22.941007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:22.941531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:22.942005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:22.943415 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:22.945114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:22.947547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 194/377] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:22.953465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:23.043780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:23.082803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25edf21-66cb-4283-a144-bb46767df2bb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:23.128706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8be4-c7b9-4aef-be35-8070c93659e0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:23.172161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:23.172832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.173268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.173547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.174382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=e2721091c11445fc8126d39812016d8f) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:23.174689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:23.176129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf51ae-d643-4491-af44-0a77bf98baa9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:23.181982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:23.182202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 195/378] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=75add8e0c0454b3a8681eab45a26244c => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:23.183312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.183446 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ca76f-7c45-4482-a6d3-b33a40663cad None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.183678 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ca76f-7c45-4482-a6d3-b33a40663cad None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.183949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ca76f-7c45-4482-a6d3-b33a40663cad None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.184374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.184865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] PATH_INFO: `/v3/groups/9c17f35fc559490299a0f3818222ba23` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.185349 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 196/379] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:23.191897 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.192084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.192353 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.194589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] RBAC: Authorizing `identity:get_group(group_id=9c17f35fc559490299a0f3818222ba23)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:23.196635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:23.200403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 184/380] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:22 2022] GET /identity/v3/groups/9c17f35fc559490299a0f3818222ba23 => generated 229 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:23.206654 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:23.209449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:23.213087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:23.215535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:23.330277 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 197/381] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:23.353604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d6347-3add-49ae-be59-3f99457ea646 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:23.362477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:23.397051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:23.398085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.398857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.399298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.400615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] RBAC: Adding query filter params (group.id=9c17f35fc559490299a0f3818222ba23, role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=5967761725a6499fb0767a5715d5e41f) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:23.401110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:23.403408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:23.411727 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 185/382] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:23 2022] GET /identity/v3/role_assignments?group.id=9c17f35fc559490299a0f3818222ba23&scope.project.id=5967761725a6499fb0767a5715d5e41f&role.id=d450d657419649a6ae78b9e8578202ca => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:23.538644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11dd9a-f1a4-4e59-9308-deb00cb4c1c0 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:23.580015 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:23.580851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.581128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.581424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.589503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:23.600535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:23.608937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:23.610066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:23.610987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 198/383] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:23.616504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:23.632969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:23.780105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e661a-2b66-48d7-a00d-2776ed3344ec None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:23.817911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:23.818978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:23.819480 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:23.819958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:23.820935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:23.821512 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:23.823147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:23.833009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 186/384] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:23.838539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:23.980700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5059ee-51ab-495f-b7b0-2abd5478f110 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:24.018088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:24.019434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:24.019937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:24.020428 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:24.022199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:24.023949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:24.026178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 199/385] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:23 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:24.031524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:24.110878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:24.158947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f119f1-a408-4478-ba1a-b2aff1962a2a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:24.200968 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119c2fb-603c-422f-be0d-250ba96b95ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:24.201892 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119c2fb-603c-422f-be0d-250ba96b95ad None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:24.202172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119c2fb-603c-422f-be0d-250ba96b95ad None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:24.202447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119c2fb-603c-422f-be0d-250ba96b95ad None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:24.203777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119c2fb-603c-422f-be0d-250ba96b95ad None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:24.205255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119c2fb-603c-422f-be0d-250ba96b95ad None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:24.207316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 187/386] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:24 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:24.213121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:24.280515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:24.329368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ef832-c5bc-4a89-b346-cdacd9a5ec40 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:24.370684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d48e2a-358a-43fb-842c-cf1e9b35af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:24.371508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d48e2a-358a-43fb-842c-cf1e9b35af59 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:24.371794 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d48e2a-358a-43fb-842c-cf1e9b35af59 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:24.372069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d48e2a-358a-43fb-842c-cf1e9b35af59 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59/groups/e5a394bca0074698a53eaf2a6167472c/roles/e7782d74fe1a4e02b57e8e7377ba7e89` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:24.374192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d48e2a-358a-43fb-842c-cf1e9b35af59 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f218d2e4a774028b7261264ea3fdd59, group_id=e5a394bca0074698a53eaf2a6167472c, role_id=e7782d74fe1a4e02b57e8e7377ba7e89)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:24.375650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d48e2a-358a-43fb-842c-cf1e9b35af59 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:24.400873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 200/387] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:48:24 2022] PUT /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59/groups/e5a394bca0074698a53eaf2a6167472c/roles/e7782d74fe1a4e02b57e8e7377ba7e89 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:24.888480 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03de36-45d3-43a6-bf65-772b21e11687 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:24.889502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03de36-45d3-43a6-bf65-772b21e11687 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:24.889978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03de36-45d3-43a6-bf65-772b21e11687 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:24.891843 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 188/388] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:24.898562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:24.899391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:24.899869 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:24.920162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:24.921421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:24.992167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc9ca2-378e-4b28-8c71-796b5ebd915f None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:24.992816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc9ca2-378e-4b28-8c71-796b5ebd915f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:24.993370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc9ca2-378e-4b28-8c71-796b5ebd915f None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:24.995206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 189/389] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:25.000485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.001028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.001538 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.006265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 201/390] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:25.031557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:25.033103 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:25.052781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.148341 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.175719 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 190/391] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:25.201220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.243427 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b30b3-dca3-4ff3-a59b-a3040069cd63 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.277731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.308040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.308873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.309168 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.309439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.327717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:25.331238 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090697d9-31d8-4e44-a02d-03be1e8c0700 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:25.336073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:25.337053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:25.338361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 202/392] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:25.344282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.430095 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.431278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.431782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.432285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.452323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad793e0-a05a-4004-bc04-2c94d0c5c82b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.456630 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:25.464741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:25.465755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:25.466728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:25.471583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 191/393] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:25.474048 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.476993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.496739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.497449 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.497735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.498526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.499088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:25.499396 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:25.501395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:25.512261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 203/394] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:25.517988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.598066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafd3a5-7eac-4cd1-bd5f-e90e3eabee00 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.622498 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b80e-c259-44fd-9c00-671c8e9228cc None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.624626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.637585 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.638684 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.639184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.639653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.641330 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:25.641853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:25.643623 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:25.653813 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 192/395] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:25.661575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.671602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.672487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.672780 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.673046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.674307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:25.675768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:25.677751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 204/396] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:25.683381 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.733583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.781299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ba8b0-0359-4dc6-970b-4e42f2e45098 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.792572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a5fd3-4e41-4d23-a893-204c534893e8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.818721 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.819965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.820441 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.821143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.822580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:25.824368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:25.826777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 193/397] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:25.832172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.834313 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.835205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.835490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.835764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.837007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:25.838538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:25.840589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 205/398] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:25.846333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:25.896777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.945231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c70504-7b5b-4e90-8695-0f0a153e33ea None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.953152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771adf-460e-4310-b435-4e6d59393159 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:25.973736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:25.986113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:25.987147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:25.987489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:25.987852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:25.989141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:25.990723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:25.993219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 194/399] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:25 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:26.000425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063103f4-e3b0-4dcd-b672-f6f0c43cd51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:26.001373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063103f4-e3b0-4dcd-b672-f6f0c43cd51a None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.001657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063103f4-e3b0-4dcd-b672-f6f0c43cd51a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.001934 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063103f4-e3b0-4dcd-b672-f6f0c43cd51a None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59/groups/e5a394bca0074698a53eaf2a6167472c/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.002140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:26.007236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063103f4-e3b0-4dcd-b672-f6f0c43cd51a None admin] RBAC: Authorizing `identity:create_grant(project_id=6f218d2e4a774028b7261264ea3fdd59, group_id=e5a394bca0074698a53eaf2a6167472c, role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:26.008699 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063103f4-e3b0-4dcd-b672-f6f0c43cd51a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:26.036916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 206/400] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:48:25 2022] PUT /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59/groups/e5a394bca0074698a53eaf2a6167472c/roles/75add8e0c0454b3a8681eab45a26244c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:26.145437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b7033-d8ab-433e-a34f-0106f6d1ff40 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:26.176338 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:26.189229 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740966c7-a90a-4cb9-83e6-0d8fb2150d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:26.190173 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740966c7-a90a-4cb9-83e6-0d8fb2150d52 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.190453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740966c7-a90a-4cb9-83e6-0d8fb2150d52 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.190724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740966c7-a90a-4cb9-83e6-0d8fb2150d52 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f/groups/e5a394bca0074698a53eaf2a6167472c/roles/e7782d74fe1a4e02b57e8e7377ba7e89` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.193039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740966c7-a90a-4cb9-83e6-0d8fb2150d52 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2721091c11445fc8126d39812016d8f, group_id=e5a394bca0074698a53eaf2a6167472c, role_id=e7782d74fe1a4e02b57e8e7377ba7e89)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:26.194556 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740966c7-a90a-4cb9-83e6-0d8fb2150d52 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:26.226618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 195/401] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:48:26 2022] PUT /identity/v3/projects/e2721091c11445fc8126d39812016d8f/groups/e5a394bca0074698a53eaf2a6167472c/roles/e7782d74fe1a4e02b57e8e7377ba7e89 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:26.409976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2fe37-28f1-454b-98d1-6a2d7d2ca3cb None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.410408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2fe37-28f1-454b-98d1-6a2d7d2ca3cb None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.410528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2fe37-28f1-454b-98d1-6a2d7d2ca3cb None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.412274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 207/402] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:26.417885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.418216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.418433 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.442548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:26.443804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:26.478899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62262-e65c-4013-912e-c7baec7eebfb None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.479086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62262-e65c-4013-912e-c7baec7eebfb None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.479352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62262-e65c-4013-912e-c7baec7eebfb None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.481007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 208/403] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:26.486860 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.489864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.489864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.512238 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:26.514453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:26.529288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 196/404] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:26.550003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:26.601764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 209/405] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:26.625473 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:26.682118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97729c1-e08b-4a6f-a1fb-05f4bdb95986 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:26.696998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:26.722271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:26.723115 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.723403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.723685 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.747133 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:26.749191 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c801-7f31-4084-98da-0a6529f01125 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:26.757654 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:26.758888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:26.759935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:26.765608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:26.774965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 197/406] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:26.777986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:26.826003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:26.827129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:26.827712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:26.828972 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:26.850558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:26.865356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:26.866583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:26.867696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:26.871626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 210/407] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:26.879069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:26.998177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2f8d-1ab2-4b45-99ed-504b43d6ecd5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.035512 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.036537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.036972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.037438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.038360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:27.038823 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.040367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.051151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 198/408] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:27.057001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.091706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:27.155433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f498ce-0c7e-40b8-a675-b21a7bba73d4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.159557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad51f7-7870-4e92-a4a7-b5ef345a5cb1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.193887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.194765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.195041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.195311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.196518 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.197989 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.204297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 199/409] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:27 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:27.205569 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.266689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.267303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.267593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.267873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.268650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:27.268963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.295270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 211/410] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:27.301352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.312558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3a768-0f7d-46c4-9bfb-14ae70de05de None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.316003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:27.348876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.349918 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.350230 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.350505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.351718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.353187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.359938 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 200/411] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:27 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:27.361272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.362513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:27.467451 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733ca8-f463-4ea5-a939-09fda4ed649b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.510252 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bd861-067f-4f1d-9f90-54575cf59601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.511785 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd861-067f-4f1d-9f90-54575cf59601 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.512214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd861-067f-4f1d-9f90-54575cf59601 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.513031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd861-067f-4f1d-9f90-54575cf59601 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f/groups/e5a394bca0074698a53eaf2a6167472c/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.515748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b21e1e-55db-49b5-99eb-ee51fb20e1fc None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.516044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bd861-067f-4f1d-9f90-54575cf59601 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2721091c11445fc8126d39812016d8f, group_id=e5a394bca0074698a53eaf2a6167472c, role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.517908 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bd861-067f-4f1d-9f90-54575cf59601 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.549874 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 201/412] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:48:27 2022] PUT /identity/v3/projects/e2721091c11445fc8126d39812016d8f/groups/e5a394bca0074698a53eaf2a6167472c/roles/75add8e0c0454b3a8681eab45a26244c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:27.552711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb48f9-40f8-41dd-b4c4-01e92f2008a2 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.552998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb48f9-40f8-41dd-b4c4-01e92f2008a2 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.554740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb48f9-40f8-41dd-b4c4-01e92f2008a2 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.557202 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 202/413] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:27.562986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.563292 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.563601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.583153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.584981 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.585337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.585616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] PATH_INFO: `/v3/groups/9c17f35fc559490299a0f3818222ba23` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.589011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] RBAC: Authorizing `identity:get_group(group_id=9c17f35fc559490299a0f3818222ba23)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.591322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.594579 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 212/414] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:27 2022] GET /identity/v3/groups/9c17f35fc559490299a0f3818222ba23 => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:27.599139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:27.600444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:27.601787 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.668419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:27.770026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 203/415] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:27.794374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.796831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:27.818357 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86575-8b59-4a17-8b73-54076722b34e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.886362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.887663 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.888268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.888804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] PATH_INFO: `/v3/projects/5967761725a6499fb0767a5715d5e41f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:27.890901 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] RBAC: Authorizing `identity:get_project(project_id=5967761725a6499fb0767a5715d5e41f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:27.892657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:27.894792 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 213/416] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:27 2022] GET /identity/v3/projects/5967761725a6499fb0767a5715d5e41f => generated 344 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:27.906173 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:27.915056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8ae3f-ace5-48c5-aca9-1b7def005277 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:27.985951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:27.986776 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:27.987066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:27.987341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:28.013465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:28.017758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57222bff-c9d9-442d-8699-be9604f29387 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:28.022010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:28.022973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:28.023862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:28.029764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 204/417] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:28.035028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:28.053744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:28.054660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:28.054945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:28.055225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] PATH_INFO: `/v3/projects/5967761725a6499fb0767a5715d5e41f/groups/9c17f35fc559490299a0f3818222ba23/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:28.060555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5967761725a6499fb0767a5715d5e41f, group_id=9c17f35fc559490299a0f3818222ba23, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:28.062043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:28.081962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:28.086441 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 214/418] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 14:48:27 2022] PUT /identity/v3/projects/5967761725a6499fb0767a5715d5e41f/groups/9c17f35fc559490299a0f3818222ba23/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:28.189512 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643bc1-88de-4509-ab37-12b983313a86 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:28.229948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:28.230683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:28.230971 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:28.231250 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:28.232021 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:28.232396 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:28.233868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:28.243458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 205/419] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:28.250516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:28.281767 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:28.387941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799df3df-82ed-43c7-8e10-435624c077f0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:28.427562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:28.428242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:28.428524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:28.428802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:28.503001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:28.504484 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:28.530091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:28.607214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 215/420] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:28.612866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:28.697659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:28.717824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f0e5e-7d44-4235-8836-3c2402f90741 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:28.757527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:28.758413 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:28.758691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:28.758960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:28.760166 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:28.761663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:28.763716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 206/421] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:28 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:28.769329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:28.909653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829826cc-38ef-409c-af9b-1f43c635540a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:28.936307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:28.959167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1b593-2826-4bba-ad12-a6a3d38e0e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:28.960058 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b593-2826-4bba-ad12-a6a3d38e0e36 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:28.960339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b593-2826-4bba-ad12-a6a3d38e0e36 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:28.960610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b593-2826-4bba-ad12-a6a3d38e0e36 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:28.965926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b593-2826-4bba-ad12-a6a3d38e0e36 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:28.967443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b593-2826-4bba-ad12-a6a3d38e0e36 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:28.969579 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 216/422] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:28 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:28.975360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:29.061912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f7e2-a782-484e-a0da-805a2deb950d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.062572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f7e2-a782-484e-a0da-805a2deb950d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.063089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f7e2-a782-484e-a0da-805a2deb950d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.064959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 217/423] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:29.070424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c26f4-55b2-4b77-8897-e3154c1c2d12 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.070986 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c26f4-55b2-4b77-8897-e3154c1c2d12 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.071486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c26f4-55b2-4b77-8897-e3154c1c2d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.088728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7c7c26f4-55b2-4b77-8897-e3154c1c2d12 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:29.090311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7c7c26f4-55b2-4b77-8897-e3154c1c2d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:29.107430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359f0c7-f70d-4010-a3b9-799dc8da863c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:29.158655 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:29.159346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.159628 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.159962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.160786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=6f218d2e4a774028b7261264ea3fdd59) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:29.161099 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:29.162647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:29.180544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 207/424] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:28 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:29.186286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 218/425] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:29.221845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:29.243579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:29.339078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ce5be-da56-4f1a-baab-3f74b629cb5a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.339670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ce5be-da56-4f1a-baab-3f74b629cb5a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.339670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ce5be-da56-4f1a-baab-3f74b629cb5a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.341070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 219/426] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:29.346991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.347518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.347952 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.354752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39492-7144-4007-819e-5a75d9aa2dca None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:29.386835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:29.392919 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:29.414082 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:29.415145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.415432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.415709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.441980 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:29.455621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:29.456781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 14:48:29.457713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:29.460537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 208/427] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:29.469568 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:29.520967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 220/428] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:29.550316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:29.563657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:29.597748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b2914-6bd4-4618-bd2e-3fdb16eb5882 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:29.639910 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:29.641367 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:29.641580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.643397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.643914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.644893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] RBAC: Adding query filter params (name=member) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:29.645450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:29.647411 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:29.657543 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 209/429] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:48:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:29.663520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:29.692986 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80aa38-0d0a-4646-a526-12c8dbd92da9 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:29.731186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:29.732408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.732906 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.733418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.745528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:29.754143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:29.755156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:29.756023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:29.758866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 221/430] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:29.767253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:29.779617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c19ab1-0e5b-4a7e-9cf3-b0e0317ca28f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:29.816110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:29.817191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.817682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.818118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.822086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:29.875565 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18830e-079d-4350-949a-e88b09c9cbd6 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:29.895136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:29.897212 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:29.905742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:29.914639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:29.915102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:29.915601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:29.916040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:29.917037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:29.917629 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:29.919225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:29.933950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 222/431] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:29.935222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.027191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 210/432] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:30.028078 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.042337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd25aa-7eae-4e81-a755-690c6a0e7846 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.073262 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.078312 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ae851-8e92-4179-9cbc-8bebc8b8814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.079272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ae851-8e92-4179-9cbc-8bebc8b8814d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.079778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ae851-8e92-4179-9cbc-8bebc8b8814d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.081052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ae851-8e92-4179-9cbc-8bebc8b8814d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.135494 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.145140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798a601-99cd-49ea-9d98-4c289c55fa5a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.156051 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ae851-8e92-4179-9cbc-8bebc8b8814d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.157993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ae851-8e92-4179-9cbc-8bebc8b8814d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.184173 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b987796-a1ff-41ca-8858-e40dbdd1c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.185049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b987796-a1ff-41ca-8858-e40dbdd1c890 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.185425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b987796-a1ff-41ca-8858-e40dbdd1c890 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.185697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b987796-a1ff-41ca-8858-e40dbdd1c890 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.187050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b987796-a1ff-41ca-8858-e40dbdd1c890 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.188507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b987796-a1ff-41ca-8858-e40dbdd1c890 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.190519 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 211/433] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:30.191838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8492aa-6261-419d-afab-447a097facaf None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.192117 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8492aa-6261-419d-afab-447a097facaf None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.192397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8492aa-6261-419d-afab-447a097facaf None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.198974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 212/434] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:30.199417 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8492aa-6261-419d-afab-447a097facaf None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.275874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 223/435] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:30.277529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.277529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.277853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.296049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:30.297272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:30.318747 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8492aa-6261-419d-afab-447a097facaf None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.321284 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.356000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.357336 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.357802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.358277 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.359875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.361774 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.363736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 213/436] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:30.364813 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.369863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.375825 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 224/437] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:30.376884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.474726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795a2d6-1e08-451c-aeb9-a1dbf6a4b8dc None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.486804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d0a04-4875-4003-9819-c539a39cdade None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.512268 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.513176 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.513547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.513854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] PATH_INFO: `/v3/projects/6f218d2e4a774028b7261264ea3fdd59` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.515097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] RBAC: Authorizing `identity:get_project(project_id=6f218d2e4a774028b7261264ea3fdd59)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.516529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.523203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 214/438] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/projects/6f218d2e4a774028b7261264ea3fdd59 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:30.523203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.527814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.528418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.528692 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.528998 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.529858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=e7782d74fe1a4e02b57e8e7377ba7e89, scope.project.id=e2721091c11445fc8126d39812016d8f) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:30.530163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.531589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.537931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 225/439] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=e7782d74fe1a4e02b57e8e7377ba7e89 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:30.546902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.570898 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.645787 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.649180 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10360574-20fe-4c98-a0b6-c16bc8bc809e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.695239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.695558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd81a1-c9d4-4f77-a8f3-484f2a842f00 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.696573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.697008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.697467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.711026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:30.720150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:30.721287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:30.722288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:30.725327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 215/440] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:30.731931 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.745450 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.746456 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.746732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.747005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.748446 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.750113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.752682 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 226/441] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:30.758593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.855793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.859970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe34a0-4656-464e-b2c5-cbf280b7f21a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.887093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b8c15-1dc7-4681-a754-79ae4fe173f4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:30.897198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.897911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.898197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.898470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.899423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:30.899726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.901110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c54c83-5ff5-4bcc-a325-370779e45a2e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.910719 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 216/442] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:30.912286 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3986e-8826-44d2-aa0d-5bcc62ad4e81 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.912551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3986e-8826-44d2-aa0d-5bcc62ad4e81 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.912824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3986e-8826-44d2-aa0d-5bcc62ad4e81 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.914326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 217/443] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:30.916302 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3986e-8826-44d2-aa0d-5bcc62ad4e81 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:30.931177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:30.936666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:30.937387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.937683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.937966 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.939266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=6f218d2e4a774028b7261264ea3fdd59) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:30.939964 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:30.942003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fdf17-e6ff-4d78-b0c3-24be34fcb426 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:30.948075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 227/444] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=6f218d2e4a774028b7261264ea3fdd59&role.id=75add8e0c0454b3a8681eab45a26244c => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:30.948075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414af1d-e772-44e1-941b-659296bd885d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:30.948075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414af1d-e772-44e1-941b-659296bd885d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:30.948075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414af1d-e772-44e1-941b-659296bd885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:30.968270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-d414af1d-e772-44e1-941b-659296bd885d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:30.969698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-d414af1d-e772-44e1-941b-659296bd885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:31.042912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3986e-8826-44d2-aa0d-5bcc62ad4e81 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.049345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 228/445] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:31.075562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414af1d-e772-44e1-941b-659296bd885d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.088710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.089424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.089768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.090048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.126832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.172262 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.173555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.201760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414af1d-e772-44e1-941b-659296bd885d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.235657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.248042 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.248860 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.249141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.249447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.262404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:31.270430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:31.272021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:31.273140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 229/446] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:31.279117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.294921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 218/447] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:48:31.300307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.371605 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.392342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29af53-c86d-4530-83d0-d3d9a159108a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.417054 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e22571-af21-4cf8-bacb-09185efebe0c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.430065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.430798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.431086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.431364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.432128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:31.432435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.433885 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.444080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 230/448] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:31.449798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.455582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.456454 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.456730 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.456996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] PATH_INFO: `/v3/projects/e2721091c11445fc8126d39812016d8f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.458255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=e2721091c11445fc8126d39812016d8f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.459691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.461406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 219/449] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/projects/e2721091c11445fc8126d39812016d8f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:31.467430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.506694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.557650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98a89-6097-4164-a55a-c8b0823ce12b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.568281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f466d0-3023-43d2-8141-08f82ac81cd8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.590013 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.590667 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.590963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.591221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.596420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.603446 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.604474 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.604748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.605019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] PATH_INFO: `/v3/groups/e5a394bca0074698a53eaf2a6167472c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.606299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a394bca0074698a53eaf2a6167472c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.607778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.609617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 220/450] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/groups/e5a394bca0074698a53eaf2a6167472c => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:31.615846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.658984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.660410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.717715 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.721181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeedc79-dfaa-49da-98b6-9be23b291278 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.754728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.755382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.755662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.755932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.756753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] RBAC: Adding query filter params (group.id=e5a394bca0074698a53eaf2a6167472c, role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=e2721091c11445fc8126d39812016d8f) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:31.757061 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.758541 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.765815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 221/451] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/role_assignments?group.id=e5a394bca0074698a53eaf2a6167472c&scope.project.id=e2721091c11445fc8126d39812016d8f&role.id=75add8e0c0454b3a8681eab45a26244c => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:31.772938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 231/452] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:31.775960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:31.841474 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:31.884950 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284681-0bab-4c29-aba1-a49a847f1b6e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:31.924685 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:31.925520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:31.925806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:31.926084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] PATH_INFO: `/v3/projects/5967761725a6499fb0767a5715d5e41f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:31.927442 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] RBAC: Authorizing `identity:get_project(project_id=5967761725a6499fb0767a5715d5e41f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:31.928948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:31.930752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 222/453] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/projects/5967761725a6499fb0767a5715d5e41f => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:31.936635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:32.040737 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e60bb-9004-43e7-ae6c-8de8def916a3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:32.074755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a097c3-a260-4d68-bc48-dcc630e0b263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:32.075616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a097c3-a260-4d68-bc48-dcc630e0b263 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:32.075948 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a097c3-a260-4d68-bc48-dcc630e0b263 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:32.076222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a097c3-a260-4d68-bc48-dcc630e0b263 None admin] PATH_INFO: `/v3/groups/9c17f35fc559490299a0f3818222ba23` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:32.077449 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a097c3-a260-4d68-bc48-dcc630e0b263 None admin] RBAC: Authorizing `identity:get_group(group_id=9c17f35fc559490299a0f3818222ba23)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:32.078905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a097c3-a260-4d68-bc48-dcc630e0b263 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:32.080503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 232/454] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 14:48:31 2022] GET /identity/v3/groups/9c17f35fc559490299a0f3818222ba23 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:32.086587 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:32.188584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c3050-8a2a-4980-96a8-6ac4827506a7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:32.217547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:32.224690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:32.225374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:32.225658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:32.225935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:32.226775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] RBAC: Adding query filter params (group.id=9c17f35fc559490299a0f3818222ba23, role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=5967761725a6499fb0767a5715d5e41f) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:32.227087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:32.228554 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2a29-1fef-4224-8980-44ac58f4fce6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:32.234651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 223/455] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 14:48:32 2022] GET /identity/v3/role_assignments?group.id=9c17f35fc559490299a0f3818222ba23&scope.project.id=5967761725a6499fb0767a5715d5e41f&role.id=d450d657419649a6ae78b9e8578202ca => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:34.871009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90d2f-cc4b-4125-944a-ad73f5919063 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:34.871940 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90d2f-cc4b-4125-944a-ad73f5919063 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:34.872407 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90d2f-cc4b-4125-944a-ad73f5919063 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:34.874367 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 233/456] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:34.879353 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:34.880151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:34.880616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:34.907132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:34.908724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:35.001027 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 224/457] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:35.036640 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae90d2f-cc4b-4125-944a-ad73f5919063 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:35.059955 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:35.158015 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae90d2f-cc4b-4125-944a-ad73f5919063 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:35.212758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:35.213626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.213907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.214183 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.231646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:35.233201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e4740180-4f77-48f8-92ec-59641810923e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:35.255875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:35.256921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:35.257938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:35.257938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:35.257938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:35.257938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:35.257938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 234/458] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:35.263035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:35.383907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d4e0-f45c-48af-a6d1-818bd793016e None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.384317 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d4e0-f45c-48af-a6d1-818bd793016e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.384501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d4e0-f45c-48af-a6d1-818bd793016e None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.386161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 235/459] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:35.399086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9245788-a1e6-45b4-99ee-6db7812e8ef2 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.399379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9245788-a1e6-45b4-99ee-6db7812e8ef2 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.399646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9245788-a1e6-45b4-99ee-6db7812e8ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.430207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a9e44-91b9-423f-b52b-7de1889e5a33 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:35.443232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a9245788-a1e6-45b4-99ee-6db7812e8ef2 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:35.444772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a9245788-a1e6-45b4-99ee-6db7812e8ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:35.467912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:35.479525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:35.480202 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.480492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.480902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.482100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:35.482463 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:35.484124 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7470-ebda-441a-afb2-bc92f89f20d8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:35.498779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 225/460] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:35.500553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2653-d497-4a4b-8f5b-dd52b6fea206 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.500841 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2653-d497-4a4b-8f5b-dd52b6fea206 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.501140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2653-d497-4a4b-8f5b-dd52b6fea206 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.503321 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 226/461] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:35.504025 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2653-d497-4a4b-8f5b-dd52b6fea206 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:35.530758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:35.545262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 236/462] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:35.545262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.545262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.545262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.559866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:35.562152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:35.648126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 237/463] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:35.650229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:35.661722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2653-d497-4a4b-8f5b-dd52b6fea206 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:35.713833 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:35.714560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.714903 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.715250 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.716152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:35.717734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:35.746171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 227/464] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:35 2022] POST /identity/v3/users => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:35.748110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:35.816135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288381df-ca1e-4ad8-9aa7-5e6660efaaab None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:35.836653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:35.859976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:35.871678 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:35.872565 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.872857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.873136 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.889566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:35.892591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:35.918199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb51e0-8c09-489a-9840-1397827d6212 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-01aa42a1-cd66-403a-b42e-d7a06d6c8793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:35.921649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:35.922614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:35.923592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:35.923592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:35.923592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:35.923592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:35.927577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 238/465] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:35.929336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38c76a-761a-4f23-a3e8-b816da5f4675 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.929865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38c76a-761a-4f23-a3e8-b816da5f4675 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.930359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38c76a-761a-4f23-a3e8-b816da5f4675 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.932541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 239/466] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:35.933611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38c76a-761a-4f23-a3e8-b816da5f4675 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:35.973210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:35.974293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:35.974588 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:35.974868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:35.993035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:35.994632 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b0eb3830-5c2b-4351-963d-39707d8dde74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:36.014375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:36.015620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:36.017219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:36.017219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:36.017219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:36.017219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:36.020849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 228/467] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:36.022374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.022934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.022934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.048896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:36.054382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:36.084324 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38c76a-761a-4f23-a3e8-b816da5f4675 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.120944 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.121803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.122091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.122804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.123823 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:36.124134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.125967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d815d-fce5-4745-96e6-6d7f46d226dd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.130380 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 229/468] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:36.131358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.137777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 240/469] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:36.142643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce271-5c51-4335-b7b7-903a75148d79 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.142643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce271-5c51-4335-b7b7-903a75148d79 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.142643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce271-5c51-4335-b7b7-903a75148d79 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.142643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 241/470] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:36.143604 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ce271-5c51-4335-b7b7-903a75148d79 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.164573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:36.191320 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:36.253779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c8003-65e8-4ac4-a670-c1980ab79358 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.256622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ce271-5c51-4335-b7b7-903a75148d79 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.292107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.292581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.292801 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.293045 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.293346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.294090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:36.294362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.294487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.294959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.295446 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.295789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9670d-26bb-4d63-b441-e75b8c07ca75 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.296599 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.298946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.307166 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 230/471] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:36.308416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.308753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.309028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.326019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:36.327233 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:36.327493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 242/472] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:36 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:36.328643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.427720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 231/473] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:36.429154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.450315 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:36.457717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bf1bc-ecc1-4707-aa98-e21390d08373 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.464428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:36.501778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.502982 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.503492 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.504054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.519515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.521397 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:36.542327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:36.543342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:36.544305 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:36.544305 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:36.544305 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:36.544305 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:36.548546 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 243/474] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:36.549633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.579869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02652b0-d73e-41fc-9f34-8d9ffe3554e7 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.624715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.628303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.628303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.628303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.628303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.628695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.665314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 232/475] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:36 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:36.667024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.673088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa92134-5c30-4d7a-ac99-3c6728778c8d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.710522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.711295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.711613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.711930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.712727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.714178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.729225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.sql.core [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:48:36.729225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:48:36.729225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [parameters: {'id': '831a2a2cac3e499e89e4d90222d62595', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:48:36.729225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=213541) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:48:36.757827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application conn.query(q) May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result.read() May 19 14:48:36.763775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return ref.to_dict() May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:48:36.764839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application next(self.gen) May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application yield resource May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application next(self.gen) May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise self.value May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application session.commit() May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:48:36.766108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.session.flush() May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application compat.raise_( May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:48:36.767241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:48:36.768253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application conn.query(q) May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result.read() May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:48:36.769154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [parameters: {'id': '831a2a2cac3e499e89e4d90222d62595', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:36.770154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 14:48:36.771581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:36.772562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 244/476] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:48:36 2022] POST /identity/v3/roles => generated 144 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:36.772562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.809623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:36.818972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f38f2-c332-4f82-8a0f-dc98100b1030 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.887160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.887847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.888153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.888427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:36.889178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:36.889531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:36.891276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:36.905708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 233/477] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:36.911377 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:36.947401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82dfe3-7cb4-4012-a74f-d617d7fec048 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:36.984167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:36.985364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:36.985853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:36.986319 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:37.000789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:37.012003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:37.013040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 14:48:37.014002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:37.015288 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 245/478] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:48:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:37.026255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:37.040322 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3965925-e75d-45ae-ae23-1b2148c85823 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:37.071248 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:37.077881 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f3cdf-726d-4183-8be6-095366fb3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:37.078373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f3cdf-726d-4183-8be6-095366fb3e86 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:37.078720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f3cdf-726d-4183-8be6-095366fb3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:37.079063 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f3cdf-726d-4183-8be6-095366fb3e86 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:37.079960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f3cdf-726d-4183-8be6-095366fb3e86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:37.082941 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f3cdf-726d-4183-8be6-095366fb3e86 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:37.121116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 234/479] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:36 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:48:37.147726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:37.199750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6edfb-5f0a-40e5-9e60-6c1ad9b2b4b1 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:37.243291 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:37.243982 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:37.244268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:37.244543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:37.245328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:37.245638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:37.247032 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5354766-e288-4224-8ac9-a67ac771092d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:37.264204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 246/480] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:38.356740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9767d4-1d5f-4be1-a908-2580ee9305cc None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:38.357350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9767d4-1d5f-4be1-a908-2580ee9305cc None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:38.357350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9767d4-1d5f-4be1-a908-2580ee9305cc None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:38.359051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 235/481] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:38.364342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:38.364766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:38.364954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:38.382695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:38.383905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:38.467627 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 247/482] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:38.498553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9767d4-1d5f-4be1-a908-2580ee9305cc None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:38.634282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9767d4-1d5f-4be1-a908-2580ee9305cc None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:38.689527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:38.690703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:38.691174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:38.691635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:38.712177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:38.720505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:38.721574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:38.722565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:38.725061 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 236/483] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:38.730782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:38.818098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:38.870150 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8970a4e-4399-4b90-b2dc-6e669b49cbd1 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:38.912969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:38.913609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:38.913887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:38.914156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:38.914915 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:38.915213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:38.916607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68035c31-cfe5-423d-982a-687958fd85d8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:38.925644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 248/484] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:38.931032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:38.999752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:39.076544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a6f47-57f2-4311-8ffb-869c01ebebb0 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.093346 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545a4ee-843f-4eb6-917e-4535c14757ae None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.093798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545a4ee-843f-4eb6-917e-4535c14757ae None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.094083 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545a4ee-843f-4eb6-917e-4535c14757ae None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.096200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 249/485] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:39.102236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.102514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.102785 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.122230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:39.124201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:39.132597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.133439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.133716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.133983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.159597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:39.161089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:39.198559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:39.199562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:39.200522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:39.200522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:39.200522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:39.200522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:39.200522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 237/486] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:39.203733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 250/487] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:48:39.210179 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.260614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.326062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:39.333134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defcbea-21c0-43e5-9dd8-6e2e5f472c68 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.386282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:39.410465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.411193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.411479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.411751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.412591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:39.413140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:39.415445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:39.436902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 238/488] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:39.442366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.449093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a324c77-c0ae-46db-b938-18d749a01861 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.486774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a59ee-abce-44db-9633-136e3cbe1813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.487594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a59ee-abce-44db-9633-136e3cbe1813 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.487877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a59ee-abce-44db-9633-136e3cbe1813 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.488152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a59ee-abce-44db-9633-136e3cbe1813 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.508564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a59ee-abce-44db-9633-136e3cbe1813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-729a59ee-abce-44db-9633-136e3cbe1813 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:39.516703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:39.517709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:39.518615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:39.529334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 251/489] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:39.530814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeac01-e87c-458e-9749-488db9b09d11 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.531109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeac01-e87c-458e-9749-488db9b09d11 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.531388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeac01-e87c-458e-9749-488db9b09d11 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.533056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 252/490] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:39.533926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caeac01-e87c-458e-9749-488db9b09d11 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.601560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9a1b2-f3a6-46bf-9b69-8a165e61a1b7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.645217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.646787 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.647583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.648016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] PATH_INFO: `/v3/users/nova` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.654436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caeac01-e87c-458e-9749-488db9b09d11 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.658426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:39.662383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:39.676105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-bf39c870-f4fd-43ac-94fb-075fdbe00093 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:39.685311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:39.686489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:39.687530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:39.690901 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 239/491] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:39.692863 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62720c0c-4175-460e-90aa-f169be261881 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.693344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62720c0c-4175-460e-90aa-f169be261881 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.694209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62720c0c-4175-460e-90aa-f169be261881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.704682 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.705317 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.705607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.705879 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.706659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:39.707059 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:39.708410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:39.718932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-62720c0c-4175-460e-90aa-f169be261881 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:39.720441 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-62720c0c-4175-460e-90aa-f169be261881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:39.721890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 253/492] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:39.722772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.733531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:39.819206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 240/493] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:39.820913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62720c0c-4175-460e-90aa-f169be261881 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.860661 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b79644-7f7a-41e9-b62f-20a6ffe2d727 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.896591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.897289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.897617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.897849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.898946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:39.899081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:39.900436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:39.930694 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62720c0c-4175-460e-90aa-f169be261881 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:39.942459 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 254/494] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:39.942459 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:39.971832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:39.972561 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:39.972843 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:39.973124 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:39.980467 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:39.984012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:39.985541 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:39.992469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-c5dd37b5-20b2-4b88-a41f-3603d9513fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:40.005098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:40.006141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:40.007123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:40.007123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:40.007123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.007123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:40.007123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 241/495] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:40.008012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fd330-5081-42ee-a75c-1f0bff623b87 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.008739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fd330-5081-42ee-a75c-1f0bff623b87 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.009015 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fd330-5081-42ee-a75c-1f0bff623b87 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.013365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 242/496] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:40.013365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fd330-5081-42ee-a75c-1f0bff623b87 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.058828 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4295b-10c1-4fa7-9741-3631f3fbf522 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.104511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.105358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.105645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.105920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.119219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:40.124710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fd330-5081-42ee-a75c-1f0bff623b87 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:40.127803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:40.128798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 255/497] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:40.130176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.169531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.170182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.170434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.170733 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.227926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.234420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67b5e9-cefe-45ca-a85a-b92e833a4b15 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.241591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.246089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.247836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.271351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.272121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.272403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.272672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.273440 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:40.273749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.275355 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfa083-3c82-443b-a052-65ea4d94e0f0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.293172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 256/498] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:40.293172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.293172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.293172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.310490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:40.311835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:40.367882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 243/499] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:40.367882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.392969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 257/500] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:40.393916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.479614 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.484080 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acd8e3-6e3d-4f32-8926-4e592ce702f5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.490085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.506090 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853084ff-4ab2-4769-aa96-f9a851c7a631 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.519544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.520237 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.520838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.521116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.521909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:40.522219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.523604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.532029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 244/501] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:40.532822 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.545373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.546198 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.546483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.546760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] PATH_INFO: `/v3/users/glance` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.553910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.555385 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:40.571968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:40.573007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 258/502] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:40.574053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.638134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2538a-54be-4c3f-beb9-377678b5f18c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.672176 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.672988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.673295 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.673570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.673715 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e3e53-892c-48bd-bdd7-6d065ecebf5b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.684193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.685802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b478301d-f4eb-4d5c-b718-d5f766b243bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:40.703280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:40.704289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:40.705316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:40.705316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:40.705316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.705316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:40.705316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 245/503] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:40.705752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c4711-74bb-4555-aed7-e18d18f6c417 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.706016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c4711-74bb-4555-aed7-e18d18f6c417 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.706296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c4711-74bb-4555-aed7-e18d18f6c417 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.707510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 246/504] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:40.708389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c4711-74bb-4555-aed7-e18d18f6c417 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.715217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.716058 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.716471 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.716735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.725653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.727422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.728896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.736142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:40.747874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:40.748859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:40.749877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:40.749877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:40.749877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.749877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:40.749877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 259/505] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:40.749877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.815224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c4711-74bb-4555-aed7-e18d18f6c417 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.847816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.848620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.848898 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.848999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698f536-8e35-461d-8703-20e1c35e0e21 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.849170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.859660 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.861125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:40.878132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:40.879218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:40.880591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:40.880591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:40.880591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:40.880591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:40.880591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 247/506] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:40.882067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:40.884767 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:40.885493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.885774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.886046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.886885 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:40.887195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:40.888697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473c14f-5be3-44dd-9464-1a4631b28abd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:40.907382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 260/507] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:40.908614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:40.908917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:40.909166 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:40.926386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:40.927578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:40.945352 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.976426 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:40.984326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69aed59-22e7-40f2-afb0-c1cfb719c707 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:40.993962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 261/508] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:40.994709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.027067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.027746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.028026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.028296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.029060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:41.029435 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.030818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.042452 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 248/509] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:41.043331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.097255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a063109-d2c9-43ea-bdf5-64d6d58924b9 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.132334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.133001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.133316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.133595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.134329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:41.134634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.136025 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.142522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff830b-33e5-4259-a5e3-d34c89fa4cc9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.147958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 262/510] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:41.148750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.169165 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.180562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.181254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.181546 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.181824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.182562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:41.182869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.184256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.195738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.195902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 249/511] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:41.201067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.248432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa6195-8541-4a2b-bd72-2e288d0ea81d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.281688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.282341 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.282624 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.282899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.298092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d4ed-a0ae-4ca9-8325-07ccf8331428 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.331175 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.331970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.332253 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.332527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.343464 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:41.350849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:41.351298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:41.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:41.353592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:41.354626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.354764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 250/512] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:41.354764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.401421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.407986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.455408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 263/513] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:41.456342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.464392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75327d0e-c1bc-487c-b017-acd663bd45a5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.498382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.499327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.499556 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.499826 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.506623 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.508073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:41.523767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:41.524906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:41.526177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:41.529694 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 251/514] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:41.530585 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.557944 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053ad40-d08d-4416-a218-d50ab70c06ed None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.592027 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.592692 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.592967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.593272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.593974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.595432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.615107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 264/515] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:48:41 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:48:41.616170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.629759 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.635454 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64e578-148b-4ca0-85e6-65deab72d8b9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.641390 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.684296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.685293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.685641 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.685989 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.694736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.696418 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:41.712729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:41.714112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:48:41.715399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:41.716273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 252/516] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:41.716937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.743636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aad613-2172-4300-961b-9cf55a42baa7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.783657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.784377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.784731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.785081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.785964 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:41.786359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.787779 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.797059 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 265/517] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:41.802781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.841468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd810a-2ced-4579-b37f-c4f4232b3476 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.874791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.880350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.881174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.881490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.881769 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.893955 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.895763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:41.913680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:41.914818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:41.918328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:41.918328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:41.918328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:41.918328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:41.918328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 253/518] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:41.918328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:41.921291 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d28dd-4b09-4f84-8022-5b7214326006 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:41.935922 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:41.957830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:41.958559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:41.958844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:41.959120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:41.959918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:41.960229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:41.962192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:41.986340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 266/519] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:41.986340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.027593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057ea5-16e2-471b-b253-4c77dd8f5d4f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.065215 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.066276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.066635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.066984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.067910 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:42.068329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.069885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.088122 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8934604-ede0-4531-9101-c1448314a7b4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.112621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 254/520] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:42.112621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.118202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.131377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.132298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.132640 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.132977 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.144101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.146242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:42.166783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:42.168006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:42.169229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:42.169229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:42.169229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:42.169229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:42.174861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 267/521] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:42.174861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.203222 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.230855 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036abf9-3e23-466d-b816-0217f61e9350 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.267719 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.268465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.268749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.269025 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.269797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:42.270103 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.271839 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.283982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 255/522] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:42.284358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a81652-d40a-4f68-b894-6d2bbccf9c71 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.284776 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.321847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.322430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.322710 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.323217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.323813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=502c9194cb0348f185dd9a5262ad1d2e) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:42.324116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.325645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.332008 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 268/523] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/role_assignments?user.id=502c9194cb0348f185dd9a5262ad1d2e&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:42.338285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.405427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35606abc-d5a0-4008-aafe-161b049c510b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.406525 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.446864 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.447732 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.448114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.448489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.452123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.493982 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff18f6-69a4-448a-b2e8-5aae14be0aae None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.532901 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.533868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.534252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.534655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.535451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:42.535854 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.537577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.547409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.549460 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.550842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 269/524] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:42.551877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.669493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 256/525] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:42.670732 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.675476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.679222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b114633-dcb8-49d8-83d0-0133c92b64ee None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.709578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.725754 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.727219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.727727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.728236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.737900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.740028 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:42.760258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:42.761547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:42.762515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:42.763290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 270/526] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:42.764491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.791009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4edf8-098d-491e-9687-6315d552325e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.830039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.831216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.831706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.832186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.840800 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.842405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:42.859000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:42.860094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:48:42.861101 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:42.864859 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 257/527] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:42.865918 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.898015 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50d14e-97d7-4d16-bdc3-76e90601c7a9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.941288 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:42.942700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:42.943192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:42.943699 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:42.956883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:42.958136 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:42.975887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:42.976883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:42.978215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:42.978215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:42.978215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:42.978215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:42.980148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:42.981889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 271/528] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:42.982802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:42.988163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd79db4-4cea-42c9-9eba-aa8184073730 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:42.998584 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:43.025729 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.026817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.027409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.027910 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.028998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:43.029565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.031476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.055725 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 258/529] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:43.056940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.092938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a86ceb-5d6b-4ff1-bc4c-a7a33381449e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.128780 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.129639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.129921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.130195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.131452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:43.131585 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.133026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.152550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 272/530] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:48:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:43.153475 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.167968 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f6fe5-ea5d-4e98-8fcb-a8d1bdf63919 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.205943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.207114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.207603 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.208118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.209151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:43.209687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.211328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.224993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 259/531] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:43.226177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.235037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:43.243718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:43.269753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f2ed1-32c1-4149-9397-189ebd6ddd5d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.308038 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.308744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.309023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.309319 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.310525 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=b81ec4ad31e04f3db722f98810813ed3) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:43.311001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.312931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.325685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 273/532] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/role_assignments?user.id=b81ec4ad31e04f3db722f98810813ed3&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:43.326757 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.337793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde31b17-6aed-4ab9-93c2-f5716387fcd1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.376197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.377337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.377827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.378333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.453838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.455660 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.475057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:43.505977 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f671fa-6b2d-4416-aabd-d2fe3bd3b145 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.573055 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 260/533] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:43.578074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.578896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.579133 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.579268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.579430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.583852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:43.598504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.600097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:43.629772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:43.630790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:43.631765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 274/534] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:43.638801 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.706823 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6abe3-9eb8-423a-932a-d785aa626e22 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.724247 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:43.754582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.755811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.756960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.757518 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.774312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.776470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:43.800011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:43.801137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:43.802172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:43.802172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:43.802172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:43.802172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:43.804906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 261/535] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:43.810165 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.839180 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cfcc9-6d68-41c2-80a0-90f1bd1241dd None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.914338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.915050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.915330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.915601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.916382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:43.916689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.918338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6786bf-7d9c-4f32-8fb8-9026241255e6 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.928587 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ab465-1b6e-4fed-9cb5-e3d2224d1c12 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:43.943356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 275/536] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:43.944720 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf98630-20c6-4c2e-9546-546a25cc898d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.945006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf98630-20c6-4c2e-9546-546a25cc898d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.950084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf98630-20c6-4c2e-9546-546a25cc898d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.951645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 276/537] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:43.952403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf98630-20c6-4c2e-9546-546a25cc898d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:43.975040 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:43.976312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:43.977130 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:43.977640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:43.978604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:43.979126 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:43.981514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268075e-f0dd-4e78-9817-4ce7e9a58865 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:43.999219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 262/538] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:44.001581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.002098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.003042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.017184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:44.027489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:44.029031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:44.056644 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:44.074931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf98630-20c6-4c2e-9546-546a25cc898d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:44.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:44.112369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.112694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.112967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.113841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=a949f55910874277bcf7bfeec95efc96) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:44.114311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:44.115931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:44.122197 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 277/539] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:43 2022] GET /identity/v3/role_assignments?user.id=a949f55910874277bcf7bfeec95efc96&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:44.124320 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 263/540] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:44.130388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.153019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.336597 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9755-fb0c-40d6-88ca-d30efb253deb None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:44.349549 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2555ed-3538-4a8c-bf54-402e60bd2a59 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:44.374251 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:44.406439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:44.407424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.407803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.408190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.424523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:44.426596 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:44.429050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:44.430007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.430382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.430764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.454169 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:44.462272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:44.463194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:44.464153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 278/541] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:44.470659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:44.472819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:44.475002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 264/542] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:44.476675 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.479304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.530254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:44.645074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcb594-127d-4869-bdb9-fb889a8f23ae None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:44.689047 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:44.689761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.690041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.690309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.691074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:44.691377 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:44.692789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65854f20-f6bb-40bc-a711-5bd9fd369f0e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:44.704401 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 265/543] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:44.710410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f43e8-e44f-4b01-b76e-5c77d8e4a768 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.710924 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f43e8-e44f-4b01-b76e-5c77d8e4a768 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.711414 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f43e8-e44f-4b01-b76e-5c77d8e4a768 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.716982 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b19941-1692-4661-9e5f-86e152f7bc72 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:44.717670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 266/544] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:44.718717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f43e8-e44f-4b01-b76e-5c77d8e4a768 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.754098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:44.754801 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.755084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.755359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.756377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:44.756695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:44.758387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6bd1f-6673-4f76-affa-3e2468523911 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:44.771359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 279/545] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:44.772920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.772920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.773189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.789789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:44.791142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:44.816426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:44.826086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:44.845978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f43e8-e44f-4b01-b76e-5c77d8e4a768 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:44.873260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 280/546] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:44.873260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.898846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:44.900246 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:44.900738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:44.901219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:44.920966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:44.922882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:44.944713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:44.945818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:44.946778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:44.946778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:44.946778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:44.946778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:44.949265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 267/547] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:44.950588 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:44.972775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d61ae0-2202-4d61-b07f-748f432b0221 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.010076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.010954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.011162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.011385 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.012229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=a9976d68323c4f2b9f33554902ee2e85) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:45.012538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.014097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.025266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 281/548] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/role_assignments?user.id=a9976d68323c4f2b9f33554902ee2e85&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:45.025266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.058417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:45.142612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ac4f5-7eb0-40d1-9177-26c2f63d74cd None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.159387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fb18d-f8c0-4345-8b0c-5a97c04033cf None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.188221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.188532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.188784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.191214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:45.191214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.191214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.220974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 282/549] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:45.227720 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.243389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:45.256206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.257314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.257606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.257880 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.280176 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:45.288578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:45.289669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:45.290570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:45.297354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 268/550] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:45.300136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.343819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:45.352989 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2b799-195c-43a9-b9b9-a2a2c2b86f79 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.399855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.400983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.401738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.402468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.418477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.419876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea2236-f6f3-4d77-b15e-72568b1cb7eb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.420536 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:45.455830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:45.457114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:48:45.458391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:45.461159 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 283/551] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:45.467166 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.480967 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.481754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.482042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.482321 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.483258 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:45.483572 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.485154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.508421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 269/552] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:45.508421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.625620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:45.633006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8478e4-292a-43ce-8559-4763cb2851e7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.672258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.673057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.673526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.673936 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.675009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:45.675385 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.676918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.697075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 284/553] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:45.702108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.727117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:45.752974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f64d-b4b9-4a27-95ed-eb529759f2c5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.796767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.800866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.800866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.800866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.812435 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.814469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:45.833867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:45.835072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:45.836228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:45.836228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:45.836228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:45.836228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:45.843146 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 270/554] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:45.844919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:45.933303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16819ddc-f631-473f-a8cd-987899447503 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.948974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52fc65-60f0-48a3-866c-1c5498328828 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:45.977616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:45.985911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.986737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.987143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.987511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:45.988256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:45.988928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:45.990720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74580ae4-e84d-440a-9ee0-0e31db216680 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:45.995040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22804d-55ab-4978-b3f4-5fad2a80e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:45.995780 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22804d-55ab-4978-b3f4-5fad2a80e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:45.996132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22804d-55ab-4978-b3f4-5fad2a80e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:45.996476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22804d-55ab-4978-b3f4-5fad2a80e1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.008547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 271/555] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:46.010226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77734fee-a850-40fb-b0e1-f1b08a686b79 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.010697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77734fee-a850-40fb-b0e1-f1b08a686b79 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.011048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77734fee-a850-40fb-b0e1-f1b08a686b79 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.013264 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 272/556] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:46.014172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77734fee-a850-40fb-b0e1-f1b08a686b79 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.085514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22804d-55ab-4978-b3f4-5fad2a80e1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.087126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22804d-55ab-4978-b3f4-5fad2a80e1a3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.099331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.151041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77734fee-a850-40fb-b0e1-f1b08a686b79 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.188911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.189773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.190056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.190423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.194310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 285/557] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:46.195634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.195924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.196204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.203281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.204970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.215650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:46.216892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:46.221364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:46.222456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:46.223475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 273/558] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:46.224778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.251056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.305329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 286/559] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:46.305329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.343904 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.360064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e619b-92a7-47bc-975a-f6f6f5383b4c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.401366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.402612 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.403051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.403536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.411953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad3720-25b8-4f72-a6ef-41539deaf44d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.416060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.417786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:46.438410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:46.439763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:46.440973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:46.440973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:46.440973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:46.440973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:46.447584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 274/560] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:46.448796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.450505 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.451192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.451475 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.451748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.452713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:46.453028 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.454633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.480129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 287/561] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:46.481184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.537200 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.564282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d060a0-8fad-43e8-a550-26e81efefc72 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.593792 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.598513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b65f49-eeb0-469f-8c44-217a743952b9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.604370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.605430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.605886 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.606310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.617316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:46.625288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:46.626293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:46.627297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:46.631847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 275/562] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:46.632596 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.643899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.644747 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.645029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.645927 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.646406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:46.646709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.648091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7aaf3-fd75-45d5-8620-41c72476409b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.661360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 288/563] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:46.662725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab75ec-b559-4d4c-89e8-1e1a1c8418e9 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.663014 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab75ec-b559-4d4c-89e8-1e1a1c8418e9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.663317 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab75ec-b559-4d4c-89e8-1e1a1c8418e9 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.664688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 289/564] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:46.669281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab75ec-b559-4d4c-89e8-1e1a1c8418e9 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.754767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dad87-bd6a-4b95-9456-516c6569d4c1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.772827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab75ec-b559-4d4c-89e8-1e1a1c8418e9 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.794036 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.795162 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.795760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.796042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.801630 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.806472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.807152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.807439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.807743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.808466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:46.808778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.810276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.812688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.814352 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.818497 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 290/565] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:46.819294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-ec5aa987-cd2c-4116-9bae-aa066dace350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:46.837790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:46.838867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:46.839824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:46.839824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:46.839824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:46.839824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:46.842206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 276/566] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:46.843993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.844449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.844875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.847377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:46.863315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:46.864521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:46.932159 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3bef-7877-49d5-9110-c8c55aef5b28 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:46.943832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 277/567] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:46.944942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:46.973098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:46.973951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:46.974234 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:46.974511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:46.981603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:46.983161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:46.997606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:46.998567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:46.999526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 291/568] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:47.000130 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.067160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909a80b-ad8b-40db-8052-fb90dc6681ca None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.085155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.090804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.107098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.107928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.108215 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.108498 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.113128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b0631-69d9-4732-826a-1ecd7fabb50c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.122073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.123956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:47.146483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:47.147697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:47.148958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:47.148958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:47.148958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:47.148958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:47.152340 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 278/569] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:47.153039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.154233 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.154934 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.155218 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.155490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.156223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.156535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.157956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.170543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 292/570] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:47.171486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.267256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8da9d4-f0c4-462d-a936-56e216f2d395 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.279611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546487-34f8-4922-83c0-a4d9e431269e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.305003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.305983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.306344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.306652 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.315504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.316172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.316455 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.316727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.317616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.317796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:47.317898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.319290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:47.325948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:47.327007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:47.327948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 279/571] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:47.328682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.333958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.341535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 293/572] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:47.342248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.362573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.439718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d84b1-c6d3-4ff6-8c6a-40162dec36e8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.451394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c48b1-8e31-4853-a997-b234f0c06cea None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.474227 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.475169 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.475455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.475730 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.476470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.476777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.478293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.489476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.490289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.490570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.490841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.495448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 280/573] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:47.495448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.502306 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.503865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:47.518199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:47.519361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:47.520873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:47.521613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 294/574] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:47.522394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.584806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.598073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.608444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe5748-631c-404f-bb99-36faeda96add None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.628362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6afe83-cfe9-4a40-b0dd-cca94298d1be None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.651521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.652213 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.652492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.652768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.653556 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.653861 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.655240 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.663839 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 281/575] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:47.663960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.664466 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.664598 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.664866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.665134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.665990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=7734eb173a1149839ce9c11ccc12013a) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.666293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.667723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.673755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 295/576] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/role_assignments?user.id=7734eb173a1149839ce9c11ccc12013a&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:47.674418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.783278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ce67-183b-449a-abdd-23bd5d62a8fe None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.796767 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf0b21-2f58-4d43-8a34-46ce94a7d1af None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.822767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.823490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.823628 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.823840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.824111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.833272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.834757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.836623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.837258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.837552 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.837829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.838633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.838947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.840401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b08c6906-5f2f-4c0e-8681-5ac6ba23fea7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:47.851526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:47.852706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 282/577] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:47.854114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00e462-7a09-404b-acc2-8d4c3e1e5f88 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.855209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00e462-7a09-404b-acc2-8d4c3e1e5f88 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.855209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00e462-7a09-404b-acc2-8d4c3e1e5f88 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.856591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 283/578] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:47.857864 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00e462-7a09-404b-acc2-8d4c3e1e5f88 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.858741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 296/579] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:47.863346 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:47.877827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:47.961214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ebf1b-4b95-42e8-baaf-d89a8bd543a3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.974007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00e462-7a09-404b-acc2-8d4c3e1e5f88 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:47.994997 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:47.995679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:47.995964 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:47.996243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:47.997089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:47.997475 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:47.998907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b122-c79f-4eff-ae78-c3cbe370d1d3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.010651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.011806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.012362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.013052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.017321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 297/580] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:48.017753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.018036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.018318 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.031699 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.033276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.037480 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:48.038668 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:48.052085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:48.053097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:48.059410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:48.059410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:48.059410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:48.059410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:48.059410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 284/581] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:48.059410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.089640 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.121562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.129539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 298/582] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:48.129539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.168992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af933f-e2c1-4477-a175-46ef7ce63119 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.205790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.206693 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.206972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.207242 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/502c9194cb0348f185dd9a5262ad1d2e/roles/931260cf51714e41884b328407aed54f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.222804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=502c9194cb0348f185dd9a5262ad1d2e, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.224244 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.243697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c41cea-83b4-4ddf-a597-08dd897dcb00 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.245326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 285/583] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:48 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/502c9194cb0348f185dd9a5262ad1d2e/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:48.248217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.295736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.296796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.297178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.297591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.310222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.312045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:48.332403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:48.333768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:48.334930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:48.334930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:48.334930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:48.334930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:48.336369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 299/584] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:48.337429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.366880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.400871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ada7b-2a56-4c13-ad23-3db4b9ee1e85 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.418516 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.447806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.450793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.450793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.450793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.451793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:48.452112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.453536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.466708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eeb56-8807-4d8f-9e1a-b2dbae1816fc None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.470258 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 286/585] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:48.471192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.514359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.515372 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.515654 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.516011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.528315 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:48.536774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:48.538057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:48.539107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 300/586] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:48.539894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.609465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5ef8-4733-4147-bee1-8cc2696ef332 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.644767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.646337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.646337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.647148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.647148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:48.648119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.648808 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.669199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 287/587] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:48.669199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.673567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.691935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d955d-17f1-4f8f-9a8c-a8a30d6ee96b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.707983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.730753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.731670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.732243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.732692 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.742141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.743653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:48.759656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:48.760731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:48:48.761802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:48.765442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 301/588] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:48.766690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.786771 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840ae2b-5a64-4b92-8ba3-028d38b2d26f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.826548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.827694 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.828202 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.828679 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.829738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:48.830255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.831969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.841439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 288/589] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:48.842494 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.886112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a908007-bf6e-44c7-a1d8-84b2e1f19c33 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:48.923080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:48.924343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:48.924623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:48.924895 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:48.932310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:48.933920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:48.950263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:48.951328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:48.952301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:48.956653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 302/590] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:48.957542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:48.962153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.964755 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:48.973805 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943297d1-2215-42f1-928a-3f797baabba1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.018358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.019049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.019335 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.019613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.020933 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:49.021263 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.022763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.042154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 289/591] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:49.043081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.076730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff407-2809-4e62-be07-354f7bf085a5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.114478 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.115286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.115568 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.115842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.127745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.129435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.149328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f4f97-66b1-4d3c-adef-ebf7d3817f43 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:49.150558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:49.151581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:49.154304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:49.154304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:49.154304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:49.154304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:49.154304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 303/592] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:49.154304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.186358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.187056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.187439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.187722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.188525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:49.188836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.190524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.203401 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 290/593] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:49.204297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.221907 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:49.225818 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:49.278313 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6ec26-2cc6-4bf1-820e-787a5ea294ce None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.316101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.316908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.317186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.317396 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231978b8-d6da-4fa8-98ae-cede8ef2f8cf None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.317530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.328447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.330083 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:49.349503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:49.350795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:49.352101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:49.352101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:49.352101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:49.352101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:49.352101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 304/594] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:49.352101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.357064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.358062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.358524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.358965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.359926 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:49.360413 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.362228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.376479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 291/595] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:49.377676 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.458119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:49.473772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d040e4-de9a-4ec2-896a-61d5ed3041a5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.485732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:49.487404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f419f2-b319-4918-b68a-e8d9d08ce0fb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.515426 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.516512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.516867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.517210 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/b81ec4ad31e04f3db722f98810813ed3/roles/931260cf51714e41884b328407aed54f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.527096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.528002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.528550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.529023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.530141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:49.530653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.532247 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.533208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=b81ec4ad31e04f3db722f98810813ed3, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.535054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.549314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 292/596] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:49.549314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.554636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 305/597] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:49 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/b81ec4ad31e04f3db722f98810813ed3/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:49.555846 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.659479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8206-77b0-499d-b85a-4aaf7c1352d2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.700608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.701479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.701761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.702035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.708458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:49.718679 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.722006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:49.747476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:49.748658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:48:49.749710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:49.750521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 293/598] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:49.758990 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.781695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3e01-a37a-4f8a-9157-86c3ddb2fece None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.857936 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.858766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.859044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.859314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.873709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.875191 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:49.901696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:49.904295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:49.906487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 306/599] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:49.912058 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:49.924540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc2cb9-395b-4472-8798-e81c998cdc95 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:49.936118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:49.994064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:49.995132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:49.995628 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:49.996174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:49.997299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:49.997844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:49.999500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d84e6-578b-4ece-a0e0-f24de83c99e0 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.018879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 294/600] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:50.025153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23c5b2-0d55-4cb5-b991-c16f6c0c09f2 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.025765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23c5b2-0d55-4cb5-b991-c16f6c0c09f2 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.026265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23c5b2-0d55-4cb5-b991-c16f6c0c09f2 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.028025 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 295/601] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:50.029019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23c5b2-0d55-4cb5-b991-c16f6c0c09f2 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.046776 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e31e1e-ed0b-48f7-84c9-6e834edbb36e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.082645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.083337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.083625 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.083903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.084700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:50.085012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:50.086502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3443a2-8e20-4d3a-be16-8c7bf68ae418 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.092993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:50.105646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 307/602] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:50.105646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.105646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.105646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.121168 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:50.122462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:50.153437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23c5b2-0d55-4cb5-b991-c16f6c0c09f2 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.191534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.192827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.193356 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.193862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.201348 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 308/603] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:50.202252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.206012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:50.207781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.221696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:50.227391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:50.228384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:50.229461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:50.229461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:50.229461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:50.229461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:50.236527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 296/604] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:50.237829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.329039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b766e2-f6a9-4bc0-8493-6fc1313108c6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.372266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.374264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.374553 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.374835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/a949f55910874277bcf7bfeec95efc96/roles/931260cf51714e41884b328407aed54f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.375234 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b4ae-d820-4890-a693-9acf153a20d1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.378826 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:50.395972 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=a949f55910874277bcf7bfeec95efc96, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:50.397887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.415744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 309/605] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:50 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/a949f55910874277bcf7bfeec95efc96/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:50.416951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.427328 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.428020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.428306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.428578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.429366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:50.429678 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:50.441276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.449552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 297/606] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:50.470687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.511728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:50.563911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52d1fc-c257-4e8f-990e-08c61b767cab None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.622381 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c43d1a-1468-4645-8909-0c14586fb7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.623230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c43d1a-1468-4645-8909-0c14586fb7bb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.623598 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c43d1a-1468-4645-8909-0c14586fb7bb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.624078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c43d1a-1468-4645-8909-0c14586fb7bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.656202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c43d1a-1468-4645-8909-0c14586fb7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-95c43d1a-1468-4645-8909-0c14586fb7bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:50.664363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:50.665618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:50.668126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:50.676691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9571d7-aa14-4a07-9c33-4c2b18376a6d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.682286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 310/607] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:50.683855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a3af0-14ed-4037-858f-224956b555ea None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.684142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a3af0-14ed-4037-858f-224956b555ea None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.684416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a3af0-14ed-4037-858f-224956b555ea None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.686197 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 311/608] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:48:50.691204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a3af0-14ed-4037-858f-224956b555ea None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.728492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.729056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.729392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.729670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.738119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:50.739744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-8ec2178c-df42-4a23-a702-fed169bdbfed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:50.757391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:50.759691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 298/609] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.762492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.764221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.787703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:50.789091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:50.809692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:50.827861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a3af0-14ed-4037-858f-224956b555ea None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:50.848257 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:50.868019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 299/610] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:50.868700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:50.873362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:50.873362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:50.873362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:50.873362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:50.873362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:50.873866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:50.874509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:50.885291 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 312/611] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:50.890186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.003887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cbf1e-01e7-4dbd-8558-00706a6b8f90 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.040670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.041378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.041653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.041919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.042624 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c6ba-8ed8-452d-9a0d-e66c6af1ea78 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.043088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:51.043088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.044527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.061573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 300/612] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:51.062527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.080331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:51.090069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.091638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.092300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.092836 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.109448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.111668 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:51.134951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:51.135970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:51.136929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:51.136929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:51.136929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:51.136929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:51.139659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 313/613] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:51.140951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.179304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b24e4-2418-4c57-9485-d5bb4b44e22b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.181422 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:51.219656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.220519 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.220893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.221278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.233224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:51.243032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:51.244276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:51.245445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:51.252089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 301/614] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:51.252089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.276493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1deb-0c0e-44a2-a9db-b111936ef1c7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.321036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.322384 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.322762 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.323130 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/a9976d68323c4f2b9f33554902ee2e85/roles/931260cf51714e41884b328407aed54f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.343614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=a9976d68323c4f2b9f33554902ee2e85, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.345228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.347948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:51.369679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 314/615] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:51 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/a9976d68323c4f2b9f33554902ee2e85/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:51.371024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.402711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9910f-33a8-419b-97d5-79ee1e9e355c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.489261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.490427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.490886 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.491327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.492680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:51.494936 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.498210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.516191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 302/616] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:51.521620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.570517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:51.615065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f7faf-bc13-4079-bb8c-ab658912357d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.668819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.673285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.673285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.673285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.673285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:51.673285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.673285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.679832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2927d7-ee34-4385-a467-698eff3d6fc8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.685261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 315/617] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:51.690216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.741657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.742472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.742763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.743032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] PATH_INFO: `/v3/users/swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.745362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:51.753397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.754848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:51.771294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:51.772367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:51.773427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 303/618] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:51.780327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.824217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797e51-cdef-487e-997f-86a51bcab7fe None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:51.869472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:51.870293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:51.870575 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:51.870857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:51.891918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:51.894462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:51.899605 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:51.916555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:51.917589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:51.918561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:51.918561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:51.918561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:51.918561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:51.918561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 316/619] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:51.923513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:51.987250 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629104d-3094-406b-8347-91dfbbe32110 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.055054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6475-332f-44b2-9b26-862b339ad65c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.074971 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.076467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.076986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.077509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.078707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:52.079232 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.080884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.093037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.093781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.094064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.094340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.095079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:52.095387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.096838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.111079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 317/620] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:52.116241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.117350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 304/621] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:48:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:52.126327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.205677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:52.210094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:52.252114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37f187-2be4-4396-a091-ccdab861fcf5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.265206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d1d2-ec8a-4d03-ab63-1934640d7261 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.290764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.292138 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.292652 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.293309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.304695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.306537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:52.325315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:52.326368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:52.327344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:52.327344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:52.327344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:52.327344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:52.329831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.330099 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 305/622] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:52.330564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.330942 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.331219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.338297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.346043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.347520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:52.381285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:52.382399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:52.383429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 318/623] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:52.390737 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.465129 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584a636-b51d-4662-9266-f6247970ad38 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.467834 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:52.506087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.506767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.507049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.507324 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.508063 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:52.508373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.509802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad3b84-bc8b-4bf7-926c-4bc827ebc064 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.526470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 306/624] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:52.526470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dbf54-7d5c-421f-be19-5e7f7ff89702 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.526470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dbf54-7d5c-421f-be19-5e7f7ff89702 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.526470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dbf54-7d5c-421f-be19-5e7f7ff89702 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.530547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 307/625] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:52.532117 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.532410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.532688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.550565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:52.551997 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:52.558118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40151d-28b9-4656-b750-ee2ca071a2a7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.579673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:52.602253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.603617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.604310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.604912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.606147 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:52.606713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.609043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.635306 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 319/627] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:52.636558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.637712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 308/627] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:52.639227 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.757037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1198b00-0392-421c-b389-28ecb218abdc None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.784772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:52.796390 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d442ff4-91bd-4775-8789-843ce86c5105 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:52.798808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.799749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.800038 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.800315 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.808853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.810698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:52.826741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:52.827769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:52.828716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:52.832928 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 320/628] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:52.833875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:52.838341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:52.840076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:52.840369 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:52.840647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:52.880200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:52.917395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:52.918884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:52.978746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fed54-d46b-4f20-a330-d6a11a8bccf8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.015281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91648fb-4cc4-4da9-8045-fab47053a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.016025 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91648fb-4cc4-4da9-8045-fab47053a465 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.016308 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91648fb-4cc4-4da9-8045-fab47053a465 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.016592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91648fb-4cc4-4da9-8045-fab47053a465 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.021751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.034487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 309/629] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:53.034673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91648fb-4cc4-4da9-8045-fab47053a465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:53.034784 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c91648fb-4cc4-4da9-8045-fab47053a465 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:53.039832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:53.040807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:53.042073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 321/630] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:53.043105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0dee-6b95-4071-bbef-9055d6f7fce0 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.043353 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0dee-6b95-4071-bbef-9055d6f7fce0 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.043645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0dee-6b95-4071-bbef-9055d6f7fce0 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.049280 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 322/631] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:53.049280 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0dee-6b95-4071-bbef-9055d6f7fce0 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.141134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b0ccb-676c-48f4-9ca2-5f515b044fac None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.145194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.153434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0dee-6b95-4071-bbef-9055d6f7fce0 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.180267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.180977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.181279 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.181647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.182351 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:53.182662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.184102 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.192649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.193920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.194366 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.194640 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.196562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 310/632] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:53.197579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.211297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.212767 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a3d3d559-761c-49e0-8f0e-ab0c95ddbec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:53.230099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:53.231235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:53.232300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:53.232300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:53.232300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:53.232300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:53.232300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 323/633] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:53.232914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.233190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.233488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.254632 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:53.256595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:53.268924 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.319387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5cc67-3ae7-4333-9102-5c2a6db1f4da None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.337278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 324/634] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:53.345513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.365042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.365852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.366207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.366555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.367387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:53.367786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.369717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.378269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 311/635] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:53.379137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.430062 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.457793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc92b6-9ce3-44af-8ab7-7ee6da1bba91 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.488396 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2220d-595f-40e3-bb1c-bd1ecb306519 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.494260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.495192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.495570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.495849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/7734eb173a1149839ce9c11ccc12013a/roles/931260cf51714e41884b328407aed54f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.505725 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.513605 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=7734eb173a1149839ce9c11ccc12013a, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.515100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.525981 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.526602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.526865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.527132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.527863 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:53.528162 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.529677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.532388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 325/636] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:48:53 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/7734eb173a1149839ce9c11ccc12013a/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:48:53.533442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.545797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 312/637] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:53.547395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.678750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4436-5c76-4412-ab2b-e6a807627b20 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.689153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d359c-ea52-44ae-ad32-b8447d78b71b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.720724 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.735499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.736908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.737466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.737995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.738586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.740142 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.742096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.742096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.753087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:53.754366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.755837 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:53.761230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:53.762294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:53.763307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 326/638] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:53.764081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:53.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:53.781521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:53.782504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:53.782504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:53.782504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:53.782504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:53.782504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 313/639] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:53.782504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.818657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:53.884554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b3bd-48aa-497a-b83d-7fbf9c871ca8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.894350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab688c3-a171-42d4-a90e-8d3ffe60a39e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:53.926256 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.927094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.927380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.927653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.930057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:53.930738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.931018 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.931289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.932058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:53.932363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.933798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.934861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:53.936842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:53.942137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 314/640] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:53.943213 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-85c3006d-9008-402b-a456-94eb5407b9e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:53.952873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:53.954240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 327/641] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8954d-78a5-4c57-8b60-7e5968077c68 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:53.955984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8954d-78a5-4c57-8b60-7e5968077c68 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:53.956639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8954d-78a5-4c57-8b60-7e5968077c68 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:53.961434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 328/642] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:53.962227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8954d-78a5-4c57-8b60-7e5968077c68 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.013135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.049306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.059986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301d6ed-b52a-4139-91f7-0e00998f767a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.072327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8954d-78a5-4c57-8b60-7e5968077c68 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.095255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.095931 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.096208 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.096476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.097344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:54.097751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.099124 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.110208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.111017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.111303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.111576 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.116672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 315/643] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:54.116672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.126735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.128194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c5452557-25cb-45b0-9751-c0dd871b5a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:54.146406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:54.147874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:54.148829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:54.148829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:54.148829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:54.148829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:54.148829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 329/644] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:54.149143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.149222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.149634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.165959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:54.167160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:54.225004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10a264-3bfb-448d-92f9-95f760b04eec None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.233783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 330/645] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:54.234619 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.269811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.270825 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.271106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.271374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.272163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:54.272469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.273914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.276029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.290588 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 316/646] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:54.290588 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.302848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.359196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030c719-2ef2-4bd6-983f-743632550c48 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.394670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbee6a3-9752-46cd-96bf-a3eb80a01fa7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.394872 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.395938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.396216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.396533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] PATH_INFO: `/v3/users/glance` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.404562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.406057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:54.424113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:54.425263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:54.426314 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 331/647] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:54.427800 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.435191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.435869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.436152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.436424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.437159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:54.437496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.438892 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.451349 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 317/648] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:54.456664 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.516128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.529162 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.530441 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa84ecd-e8bc-4344-8133-cc126cb5ee18 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.561782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5be6f-315e-4762-a498-4a83715a47f2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.566903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.567722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.568007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.568486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.580608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:54.588479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:54.589501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:54.590408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 332/649] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:54.591181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.602154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.602972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.603181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.603451 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.604289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=502c9194cb0348f185dd9a5262ad1d2e) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:54.604601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.606058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.621464 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 318/650] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/role_assignments?user.id=502c9194cb0348f185dd9a5262ad1d2e&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:54.623527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.713318 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e16f-65af-4f03-844a-9519a9ceab50 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.760425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.762205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.762793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.763512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.764864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:54.765875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.767876 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.768762 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa34f9-77c5-4b6d-aab8-bf0934ba1543 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.798948 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 333/651] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:54.800041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.811175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.820408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.821358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.822056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.822337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.824523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:54.835341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.837035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:54.854358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:54.855429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 319/652] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:54.856487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:54.934097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b990-aef3-40c5-816a-4b20dc2f7f7c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.973756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:54.974439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:54.974723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:54.975003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:54.975760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:54.976068 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:54.977500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:54.986495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc5380-689d-4264-a967-6cae58f66331 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:54.990679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 334/653] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:54.991635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.032450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.033537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.034040 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.034534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.073778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.102979 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.112624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.114266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.121154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4c096-b417-46eb-a790-1855da9b84bf None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.156859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.157689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.157917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.158241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.159093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:55.159404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.160851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.184195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 335/654] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:55.184195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.259265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 320/655] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:55.259265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.296959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f5e6d-e4c4-412e-9959-a153ac912ed4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.325184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.335672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.336484 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.336768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.337042 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.348681 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.350183 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:55.370074 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:55.371125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:55.372129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:55.372129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:55.372129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:55.372129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:55.376102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 336/656] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:55.376996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.386064 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.395845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01723b1-53e6-40d4-ae4c-3cf8076e8537 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.446441 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.447097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.447379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.447650 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.510321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984846c-0256-4b8e-941d-9a78a0663021 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.519962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.522043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.560950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.563223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.563725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.564215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.578595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.580688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:55.605186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:55.606245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:55.607249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:55.607249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:55.607249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:55.607249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:55.614311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 337/657] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:55.615340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.631633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.641303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 321/658] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:55.642455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.652593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.729362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79e001-8614-44d7-905e-697f2c8c2da0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.754191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74a433-05ee-4ead-9289-28eba7f375de None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.766410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.767239 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.767592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.767960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.768799 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:55.769194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.771250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.784332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 338/659] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:55.784332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.797854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.798807 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.799292 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.799727 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.800751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:55.801270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.802851 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.815841 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 322/660] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:55.816920 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.876419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.883890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:55.894776 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7865449-692f-4445-8c5e-1d9573873dad None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.927735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38180f-532f-49cb-b89c-7144519d57e7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:55.931172 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.931989 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.932271 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.932543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.943392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.944844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:55.962177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:55.963163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:55.964128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:55.964128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:55.964128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:55.964128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:55.964128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 339/661] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:55.964961 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:55.972059 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:55.972884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:55.973167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:55.973472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:55.981861 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:55.983406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-fb8bb561-8d06-4ac4-98a1-792440905f7a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:55.997966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:55.999004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:56.000020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 323/662] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:56.001387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee71ba-b348-4d13-a6d5-9a5a22aff83a None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.001387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee71ba-b348-4d13-a6d5-9a5a22aff83a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.001387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee71ba-b348-4d13-a6d5-9a5a22aff83a None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.002488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 324/663] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:56.003220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee71ba-b348-4d13-a6d5-9a5a22aff83a None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.076819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ad2a-405e-47f2-a7e7-d2ac8066be09 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.112813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.113641 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.113927 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.114196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.114985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee71ba-b348-4d13-a6d5-9a5a22aff83a None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.121055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.122575 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.128712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-46c3da0a-ea64-4e7a-8a1e-9205fc2ceff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:56.136962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:56.138151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:56.139125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:56.142975 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 340/664] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:56.144105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.144384 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.144832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.150007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.156906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.157622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.157905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.158183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.158929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:56.159238 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.160622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.164243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:56.165448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:56.178668 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 325/665] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:56.178668 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.241309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 341/666] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:56.241309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.296071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf18ac3-ef10-4bb5-bfc8-ba7aa6df2737 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.333672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.334449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.334805 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.335248 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.336198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:56.336579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.337861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908713a9-b367-410c-99b6-8d2bb9735cdb None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.338180 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.356716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 326/667] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:56.357566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.373832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.379551 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.380590 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.380883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.381157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.382029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:56.382309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.383764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.389787 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.401401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 342/668] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:56.401401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.464194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029c42e-6ba1-4c68-a697-986cd8facd85 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.499780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.500597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.500889 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.501146 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.507671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d7c09-99af-478d-9bfe-f2fb3a7970ec None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.508652 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.510426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:56.525772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:56.526743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:56.527720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 327/669] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:56.528278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.547004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.547816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.548098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.548372 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.562385 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:56.570334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:56.571373 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:48:56.572274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:56.576497 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 343/670] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:56.577376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.623778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.638743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.640241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50ddcd-3623-4d64-bccd-4eff2f7ff9b6 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.676841 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.677545 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.677828 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.678188 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.689479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c83ec-fab5-4582-8141-ce0da2119667 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.724380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.725050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.725362 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.725641 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.726476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=b81ec4ad31e04f3db722f98810813ed3) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:56.726892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.728530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.745359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 344/671] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/role_assignments?user.id=b81ec4ad31e04f3db722f98810813ed3&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:56.745359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.762331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.763996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.879651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b081e2b-3fc1-4515-9baf-e27f136f2235 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:56.883987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.896796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 328/672] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:56.898215 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:56.900234 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:56.922364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:56.923144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:56.923502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:56.924313 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:56.924744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:56.925127 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:56.926764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:56.946890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 345/673] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:56.948046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.016707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f59492-a302-4bd1-805f-193dfd142f5e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.052990 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.053804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.054160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.054509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.055334 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:57.055454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b44d5b-1a06-452d-ad0e-14a33952e989 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.055749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.057194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.065780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 329/674] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:57.066741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.096984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.097831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.098114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.098386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.108711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.110201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.126157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a715779f-de10-4bcb-9fea-9434f6317b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:57.129114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:57.130113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:57.131078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:57.131078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:57.131078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:57.131078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:57.131078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 346/675] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:57.131992 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e2d93-030b-4fa2-8eef-9dce02662441 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.132281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e2d93-030b-4fa2-8eef-9dce02662441 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.132574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e2d93-030b-4fa2-8eef-9dce02662441 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.138385 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 347/676] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:57.138385 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e2d93-030b-4fa2-8eef-9dce02662441 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.166260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.185493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367ac2-bd90-4817-8889-1032b29e927c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.223249 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.223923 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.224525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.224817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.225942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=a949f55910874277bcf7bfeec95efc96) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:57.226259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.228222 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.237416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 330/677] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/role_assignments?user.id=a949f55910874277bcf7bfeec95efc96&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:57.238271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.254681 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e2d93-030b-4fa2-8eef-9dce02662441 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.304549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.306432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.306957 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.307751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.323583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.325684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-2e3043f8-c465-4b0f-a814-bf15fea3acec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:57.350923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:57.351979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:57.352932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:57.352932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:57.352932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:57.352932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:57.359709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 348/678] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:57.359709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.359709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.359709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.371921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c88e2b-6c9f-4f24-a3ff-282d7bd59f7d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.390152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:57.392148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.393474 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:57.413969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.414668 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.415008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.415286 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.416383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:57.416383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.417829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.437418 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 331/679] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:57.437418 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.492943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 349/680] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:57.493935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.506993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.564001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae963c-8a99-4900-b181-877d9887bee1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.610199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.610819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea35dcc-adaf-4886-89be-f1fab961259e None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.611400 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.611846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.612276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.613161 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:57.613923 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.616402 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.630817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 332/681] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:57.631994 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.662738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.663755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.664175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.664549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.674393 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.675965 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.682044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:57.692845 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:57.693889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:57.694843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:57.701684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 350/682] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:57.702832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.748086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15910f1-a266-4672-81d2-ae8483f4ed2f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.779124 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.787648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.788522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.788809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.789084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.801841 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:48:57.810412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:57.811473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:57.812974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 333/683] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:57.814376 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.820560 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b0a44-9292-4c3e-9a63-626d89c8fb5e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.857377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.858637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.859133 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.859773 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.868842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.870809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:57.886802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:57.887945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:48:57.889003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:57.896657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 351/684] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:57.897880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:57.919811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:57.923416 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e44d03-0108-4943-83c5-efb65b3af863 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:57.961995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:57.962569 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:57.962827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:57.963102 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:57.963921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:57.964235 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:57.966249 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:57.984959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 334/685] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:57.984959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.022357 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a010c35-72d9-4105-a0a3-54e32c71dd80 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.054597 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.061399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.062584 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.063120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.063400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.064164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:58.064474 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.065893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.075634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 352/686] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:58.076651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.095749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc293c-117c-4a9d-90a4-ff365dc6400e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.132213 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.133370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.133914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.134410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.135482 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:58.135997 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.137951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.162478 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 335/687] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:48:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:58.163688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.187331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.191215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c272bb-9c8d-4ecb-bbf1-401fcca79339 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.237343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.237343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.237343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.237343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.237806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=a9976d68323c4f2b9f33554902ee2e85) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:58.237806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.237806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.243658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 353/688] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/role_assignments?user.id=a9976d68323c4f2b9f33554902ee2e85&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:58.244555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.287889 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0a166-ba2b-4d2e-ba8e-5e18f4e19181 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.312466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.327453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.328596 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.329198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.329652 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.341656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.343534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:58.363271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:58.364372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:58.365570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:58.365570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:58.365570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:58.365570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:58.365842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68da6-06c6-43ed-b6fd-3a010ff28f6f None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.367957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 336/689] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:58.374063 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.402858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.403657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.404033 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.404376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.462384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.482518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.484120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.511071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262fc2a-3e0c-484c-9dbd-b0a0264a3ac9 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.553722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.555081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.555598 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.556375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.557359 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:58.557900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.559619 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.576530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 337/690] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:58.583302 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.587056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.656847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 354/691] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:48:58.657973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.721152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e446b0-de65-47af-8cb0-33e72d20497d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.750058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.758518 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.760214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.760214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.760214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.768763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.770697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.780724 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef097e8-889b-4d23-8225-d7a644baa5aa None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:48:58.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:58.787437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:48:58.788448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 338/692] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:48:58.793670 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.818404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.819282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.819567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.819839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.831677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.833204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:58.852470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:58.853602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:58.854612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:58.854612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:58.854612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:58.854612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:58.854612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 355/693] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:58.859877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:58.873777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:58.922423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f04469-c484-4b26-9d7a-92383be527ce None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:58.959454 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:58.960446 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:58.960939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:58.961419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:58.962496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:58.963028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:58.964650 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:58.983270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 339/694] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:48:58.988959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.000414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f961236-2b6b-4f6f-ae22-45b4a9c06be7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.040480 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.052144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.053365 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.054812 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.055275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.057405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:59.057905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.059869 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.083070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 356/695] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:59.092691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.148043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db80a7-49f4-43f4-a489-9e221fd2895b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.202604 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.226642 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd805c-aa9c-4299-946c-09dc38b12460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.227304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd805c-aa9c-4299-946c-09dc38b12460 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.227582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd805c-aa9c-4299-946c-09dc38b12460 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.227850 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd805c-aa9c-4299-946c-09dc38b12460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.262699 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1fb7a-777c-46e2-907c-ba3b8994a808 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.329504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.330634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.330927 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.331208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.340437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.342769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.351431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd805c-aa9c-4299-946c-09dc38b12460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.353086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd805c-aa9c-4299-946c-09dc38b12460 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-2291aafd-ba81-442c-8a02-b6706abceef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:48:59.361195 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:59.362252 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:48:59.363202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:59.363905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 357/696] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:48:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:59.364788 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0bb7-56f0-46bd-bde6-cfb47b690b58 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.365041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0bb7-56f0-46bd-bde6-cfb47b690b58 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.365361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0bb7-56f0-46bd-bde6-cfb47b690b58 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.366969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 358/697] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:48:59.369827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa0bb7-56f0-46bd-bde6-cfb47b690b58 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.422950 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.499712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 340/698] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 14:48:59.501225 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.501697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.501936 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.506094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa0bb7-56f0-46bd-bde6-cfb47b690b58 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.519786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:48:59.521488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:48:59.539914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.551223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.552435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.553320 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.553918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.555019 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:59.555708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.558027 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.576793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 359/699] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:59.577736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.608645 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 341/700] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:48:59.610146 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.707121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530ddbe-7cf2-4421-8309-9e06072847f6 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.708114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.734189 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ff202-dcb4-4fd5-9770-d0fd72a31c71 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.751665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.753120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.753632 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.754126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.768927 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.770134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.771198 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.771481 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.771758 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.772120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.772599 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=7734eb173a1149839ce9c11ccc12013a) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:48:59.772927 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.774434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.785540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 342/701] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:48:59 2022] GET /identity/v3/role_assignments?user.id=7734eb173a1149839ce9c11ccc12013a&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:48:59.786522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:48:59.792712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:48:59.793725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:48:59.794920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:48:59.794920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:48:59.794920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:48:59.794920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:48:59.798106 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 360/702] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:48:59.809862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:48:59.877821 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.936362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ad872-caff-411c-b1b4-144db8807c9f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.950888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cae0f-4979-4b73-b871-08d280aeed5c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:48:59.976488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.977462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:48:59.977748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:48:59.978024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:48:59.989749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:48:59.992589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:48:59.994083 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:48:59.998456 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:48:59.999859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.000333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.000803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.001589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:00.001896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.003284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fe903-7c8b-4ff7-8884-df5ccba3003e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:00.017073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:00.018151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:00.019617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:00.019617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:00.019617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:00.019617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:00.020090 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 361/704] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:00.020181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 343/704] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:00.020256 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4df9-a829-4529-9d6e-8a6f7dba366f None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.021832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4df9-a829-4529-9d6e-8a6f7dba366f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.021832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4df9-a829-4529-9d6e-8a6f7dba366f None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.023483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 362/705] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:00.024664 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.031527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.031817 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.032096 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.052585 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:00.054150 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:00.135350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 363/706] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:49:00.136538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.181672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:00.252912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5920580-bd69-41c4-8cab-65b19181ce73 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.281577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5eb94-04eb-450f-b98c-bd2c07ab4f36 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.291288 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.292351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.292837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.293391 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.294409 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:00.294926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.296641 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.309761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 364/707] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:00.311014 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.319193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.320013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.320290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.320562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.328214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.329811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:00.344984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:00.346019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:00.347012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:00.350974 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 344/708] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:00.351832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.410189 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:00.418570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:00.434229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b267e0b-78ae-4560-9dcb-3ef6f264c455 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.468214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30b8c2-2756-419e-ad2a-d7a42c37dbc3 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.482160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.483954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.484707 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.485207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.486378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.488298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.504265 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.505154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.505563 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.505848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.506742 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.508386 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:00.508880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:00.510002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:00.510910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:00.514430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 365/709] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:00.515484 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.541261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 345/710] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:49:00 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:00.541261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.727442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd793-2612-441c-8398-4a3f04c905d9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.769109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.770298 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.771258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.771681 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.772638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:00.773100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.773629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cd9ed-0161-43ae-9153-dc3d854a758b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.774938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.791071 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 366/711] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:00.799017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.799641 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:00.820328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:00.839196 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.840368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.840979 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.841486 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.842413 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:00.842940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.844798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.853658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 346/712] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:00.860921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:00.927801 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73357c90-54b2-425b-971c-81f76868b2fb None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.973094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037b871-40d1-4798-a107-e14fb02d88e4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:00.975669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:00.976357 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:00.976639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:00.976911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:00.978728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=502c9194cb0348f185dd9a5262ad1d2e) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:00.978994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:00.980462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3faecf-0712-4921-9b0e-c1f86a926d79 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:00.987696 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 367/713] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/role_assignments?user.id=502c9194cb0348f185dd9a5262ad1d2e&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:01.024705 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:01.025453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:01.025738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:01.026011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:01.026744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:01.027050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:01.028543 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b6377-546c-4733-a8f3-90f703dcd2f6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:01.037146 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 347/714] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:49:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:01.153607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d88f0-3b94-4d7b-b159-661108e23230 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:01.154080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d88f0-3b94-4d7b-b159-661108e23230 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:01.154546 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d88f0-3b94-4d7b-b159-661108e23230 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:01.157308 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 368/715] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:01.165650 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:01.165650 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:01.165650 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:01.189448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:01.191109 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:01.227922 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:01.284677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 348/716] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:49:01.304082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d88f0-3b94-4d7b-b159-661108e23230 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:01.392220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:01.430940 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d88f0-3b94-4d7b-b159-661108e23230 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:01.476781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:01.477626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:01.477915 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:01.478188 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] PATH_INFO: `/v3/services/network` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:01.478851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:01.480234 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:01.493676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:01.494766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:01.495662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:01.503048 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 369/717] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:01 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:01.504717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:01.653155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ce70f-df5c-47f4-b4dc-8cc73ee42bc9 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:01.701029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:01.702134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:01.702658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:01.703151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:01.704094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] RBAC: Adding query filter params (name=network) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:01.704608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:01.706246 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0badd-6704-47ff-a8a4-be9e60f3f580 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:01.711781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 349/718] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:01.718522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:01.894503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fed11-54b2-4faa-a133-37ef8c54b58d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:01.939352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:01.940048 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:01.940335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:01.940611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:01.941360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] RBAC: Adding query filter params (type=network) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:01.941670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:01.943058 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec86b6-103b-421b-960c-4e82ef646434 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:01.954799 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 370/719] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:02.221355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c20edd-4071-478a-b05f-5c61c076b3f4 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:02.221567 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c20edd-4071-478a-b05f-5c61c076b3f4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:02.221858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c20edd-4071-478a-b05f-5c61c076b3f4 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:02.223498 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 350/720] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:02.229535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:02.229821 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:02.230084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:02.242490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:02.248824 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:02.250924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:02.323134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 371/721] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:49:02.346742 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c20edd-4071-478a-b05f-5c61c076b3f4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:02.380394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:02.460282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c20edd-4071-478a-b05f-5c61c076b3f4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:02.497806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:02.498599 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:02.498865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:02.499127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:02.510262 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:02.518171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:02.519174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:02.520084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 351/722] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:02.526049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:02.675736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44edfa4-29bc-4db8-a462-c0a775b0f769 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:02.723533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:02.724214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:02.724532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:02.724772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:02.725556 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:02.725865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:02.727268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:02.734464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:02.741719 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 372/723] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:02.747002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:02.899855 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:02.910909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4cd67-2ea5-45ea-907d-75910e2df62f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:02.948797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:02.949636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:02.949911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:02.950245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:02.961365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:02.962824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:02.982797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:02.983819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:02.984777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:02.984777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:02.984777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:02.984777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:02.984777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 352/724] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:02.989947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:03.113861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69424456-6cff-4e36-9862-512597ea2a72 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:03.167983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:03.168593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.168920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.169192 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.169953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:03.170254 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:03.171644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:03.192684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 373/725] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:03.198151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:03.356083 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eecc8f-4da3-4649-9d65-4526f4e6d84a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:03.403860 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:03.404677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.404951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.405222 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.420113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:03.421507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:03.440392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:03.441577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:03.442683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 353/726] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:03.447896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:03.559656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:03.619990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cf854-950b-40b4-875c-cde44286cffe None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:03.683177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:03.684263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.684804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.685309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.686379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:03.686914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:03.688573 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f38ba-ff26-4fc1-b9ef-85d346fbd3c3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:03.714401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 374/727] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:49:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:03.719636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:03.727862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:03.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a5bf3-8743-40d5-b733-54878da82fc4 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.755776 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a5bf3-8743-40d5-b733-54878da82fc4 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.756267 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a5bf3-8743-40d5-b733-54878da82fc4 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.758339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 375/728] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:03.771496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025bfaf-4a66-4de6-b9a7-a0c33e6eb115 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.771843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025bfaf-4a66-4de6-b9a7-a0c33e6eb115 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.772048 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025bfaf-4a66-4de6-b9a7-a0c33e6eb115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.793968 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-2025bfaf-4a66-4de6-b9a7-a0c33e6eb115 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:03.795508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-2025bfaf-4a66-4de6-b9a7-a0c33e6eb115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:03.872100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93746e-e4dd-4272-8324-542a382f1b4e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:03.889270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 376/729] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 14:49:03.889270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22ad48-97d6-4842-a8ea-2e5ecee3bbca None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.889270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22ad48-97d6-4842-a8ea-2e5ecee3bbca None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.889270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22ad48-97d6-4842-a8ea-2e5ecee3bbca None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.889270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 377/730] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:03.892688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.892978 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.893269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.923067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:03.924837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:03.928626 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766fc61-9951-472a-98ce-57612f0b9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:03.929623 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766fc61-9951-472a-98ce-57612f0b9272 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:03.930183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766fc61-9951-472a-98ce-57612f0b9272 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:03.930678 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766fc61-9951-472a-98ce-57612f0b9272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:03.977656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.003685 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766fc61-9951-472a-98ce-57612f0b9272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.005187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766fc61-9951-472a-98ce-57612f0b9272 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.013283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 378/731] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:49:04.013449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.013549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.125913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 354/732] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 14:49:04.127651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f4b0-f53a-4db1-b3f9-003a264275aa None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.128103 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f4b0-f53a-4db1-b3f9-003a264275aa None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.128534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f4b0-f53a-4db1-b3f9-003a264275aa None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.130495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 355/733] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:04.131447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f4b0-f53a-4db1-b3f9-003a264275aa None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.250024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f4b0-f53a-4db1-b3f9-003a264275aa None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.251589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.257725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe09aca-21f4-47ba-86de-07df36625fd4 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.287651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.288460 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.288741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.289011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.295138 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e60fc9-ebdf-44c6-acfa-b14bd873118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.295952 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e60fc9-ebdf-44c6-acfa-b14bd873118a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.296237 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e60fc9-ebdf-44c6-acfa-b14bd873118a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.296507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e60fc9-ebdf-44c6-acfa-b14bd873118a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.300227 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:04.307231 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e60fc9-ebdf-44c6-acfa-b14bd873118a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:04.308032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:04.309032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:04.310331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 356/734] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:04.311087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-20e60fc9-ebdf-44c6-acfa-b14bd873118a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:04.322200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:04.323226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:04.324151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 379/735] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:04.325382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.325382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.325597 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.343791 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:04.344990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:04.375000 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.416440 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 380/736] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:49:04.417342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.432858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432fbf-e630-41c8-b41c-fdc32c26e289 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.479602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457909cf-9267-4654-b136-10ae51584291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.481365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457909cf-9267-4654-b136-10ae51584291 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.481840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457909cf-9267-4654-b136-10ae51584291 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.482331 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457909cf-9267-4654-b136-10ae51584291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.496855 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457909cf-9267-4654-b136-10ae51584291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.498789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457909cf-9267-4654-b136-10ae51584291 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.517405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce5567-6b66-41a8-b5f5-2aaf755e2b19 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-457909cf-9267-4654-b136-10ae51584291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:04.522697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:04.523657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:04.530043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:04.530043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:04.530043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:04.530043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:04.530043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 357/737] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:04.530043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457909cf-9267-4654-b136-10ae51584291 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.545756 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.555974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.556837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.557127 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.557450 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.558186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:04.558649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.560006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.569356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 381/738] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:04.570049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.599187 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.652004 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457909cf-9267-4654-b136-10ae51584291 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.681807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37368f-ed08-4128-b5c5-42f042bb386d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.690467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.691236 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.691520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.691797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.692684 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:04.692999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.694459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.709984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 358/739] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:04.709984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.716319 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.717011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.717290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.717588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.718315 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.719672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.736036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 382/740] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:04 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:04.737851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.807932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.845658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f315205-ec54-4af6-89d0-03af209d3032 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.868410 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:04.893531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928acbe-6339-4c05-9b4f-69a6f4597019 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:04.898201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.902153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.902826 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.904187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.905285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:04.905825 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.908748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.930077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 359/741] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:04.931700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:04.939729 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:04.940627 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:04.940889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:04.941186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:04.954018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:04.955496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:04.975600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:04.976630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:04.977633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:04.977633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:04.977633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:04.977633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:04.977633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 383/742] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:04.977959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.045858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef32a77-88bd-4d0e-8f11-0f67ef1ee0ff None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.085865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.087054 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.087744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.088977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.092479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.101731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.104039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.114037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5516d-9c8b-40b8-9ecc-0a8ab0cbd2b3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:05.122374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:05.123355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:05.124329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:05.124329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:05.124329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:05.124329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:05.124329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 360/743] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:05.125502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.155481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.156308 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.156591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.156864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.159775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.165075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.166904 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-2ec2dee1-ca9b-47cd-b555-c15bf7264179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:05.181847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:05.183017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:05.183983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 384/744] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:05.184687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe347a-dcdf-4357-8bab-5434f8f8d0ca None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.184970 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe347a-dcdf-4357-8bab-5434f8f8d0ca None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.185260 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe347a-dcdf-4357-8bab-5434f8f8d0ca None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.189358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 385/745] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:05.189358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe347a-dcdf-4357-8bab-5434f8f8d0ca None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.238752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2701-ba5e-4f8d-893b-b8051e35074b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.275420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.276128 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.276404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.276675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.277461 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:05.277765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.279218 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.297333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 361/746] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:05.297333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.304628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe347a-dcdf-4357-8bab-5434f8f8d0ca None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.344619 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.345210 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.345539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.345813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.346551 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:05.346857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.348236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720f5d-4e74-41eb-aa6d-550c64c813e0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.357151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.361853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 386/747] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:05.363307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.363595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.363875 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.383671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:05.384880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:05.416474 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c86c70-6e13-4b1e-81a3-2ffbf4237f41 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.438797 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.451849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.452527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.452811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.453085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.453915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:05.454226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.455662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.461416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 387/748] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 14:49:05.461416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.473050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 362/749] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:05.478616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.569449 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc91ea1-9043-43fd-b942-888ac9e56526 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.586243 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673c053-43f8-4c82-8efb-86ade64f13e0 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.596854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.607135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.607963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.608245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.608515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.616093 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.617591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.623779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.624602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.624875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.625147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] PATH_INFO: `/v3/users/nova` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.632903 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.634377 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:05.637751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:05.638740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 388/750] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:05.642555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:05.657635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:05.658735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 363/751] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:05.659767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.695053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.751783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6355c1c-97ac-425b-9bd3-dd2767b88a54 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.769673 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac40b4-f76c-4d1a-980d-665bb88ac593 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.787052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.787723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.788010 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.788285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.789119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=7734eb173a1149839ce9c11ccc12013a) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:05.789463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.790921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.805273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 389/752] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/role_assignments?user.id=7734eb173a1149839ce9c11ccc12013a&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:05.805273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.815464 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:05.816184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:05.816436 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:05.816711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:05.817622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:05.819159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:05.838767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 364/753] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:05 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:05.839968 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:05.877738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:05.969903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaee32-a3f3-4fa4-82eb-539c059d902d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:05.994531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:06.058998 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.059785 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.060099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.060333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.061212 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:06.061539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.063063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:06.079826 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec5102-8a4a-486f-99db-f1b590019271 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.101660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 390/754] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:06.113217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:06.126997 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.128354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.128789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.129221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.130868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:06.131327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.133093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:06.160344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 365/755] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:49:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:06.164926 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:06.257741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45328f-b6d5-4d9d-ab20-f2ccdcf14972 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.280655 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:06.318398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b226b6-41e3-4e3b-85d0-6cecdc42b7de None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.322629 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.323684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.324294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.325052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.375869 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:06.389506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.390269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.390557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.390830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.411244 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.412720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:06.418505 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.420513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:06.441716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:06.442773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:06.443751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:06.443751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:06.443751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:06.443751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:06.443751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 366/756] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:06.448832 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:06.541793 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 391/757] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:06.547594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:06.613018 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c016af5-5f02-45cb-9dc4-37ff269a5f29 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.650527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:06.694584 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276dc43-9084-44a7-bb0a-7a9972a276bf None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.696753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.697469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.697754 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.698032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.698781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:06.699090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.700566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:06.702422 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:06.732056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 367/758] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:06.735481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.736919 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.737473 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.737758 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:06.738197 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.760201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.762634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:06.789507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:06.790564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:06.791537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:06.791537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:06.791537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:06.791537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:06.792419 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 392/759] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:06.800826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:06.952757 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3fd0a-4ff6-43f3-b4bb-23139214908b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.977738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437014-7083-4ff7-9e13-32eff7f7d864 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:06.987942 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:06.990171 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:06.990859 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:06.991134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:06.991416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:06.992221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:06.992526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:06.994513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.012404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 393/760] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:07.018086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:07.067120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.068381 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.069688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.070191 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.088182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.091298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:07.121540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:07.122666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:07.123665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:07.130245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 368/761] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:07.136879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:07.219019 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed3656-f8f2-4c0a-b1aa-ff39270263ed None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:07.249439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:07.263689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.264524 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.264813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.265086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.273983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.275454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9825967f-0086-42ae-8a5b-b5892b19cba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:07.290064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:07.294711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:07.295974 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 394/762] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:07.296729 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083414-1f83-4dd7-8398-a94f8c9df8cd None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.296898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083414-1f83-4dd7-8398-a94f8c9df8cd None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.299671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083414-1f83-4dd7-8398-a94f8c9df8cd None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.304137 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 395/763] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:07.304283 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c96e-b744-4c83-ab0f-88c794c589f1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:07.304932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80083414-1f83-4dd7-8398-a94f8c9df8cd None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:07.341873 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.342677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.343021 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.343364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.344259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:07.344639 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.346289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f66ba8-0611-44e9-8ea5-393a1d3fd86b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.346687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:07.363342 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 369/764] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:07.368673 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.369050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.369517 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.387211 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:07.388660 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:07.420297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80083414-1f83-4dd7-8398-a94f8c9df8cd None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:07.455914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.456593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.456879 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.457153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.457986 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:07.458293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.459865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.462398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 370/765] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:07.463322 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:07.485953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 396/766] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:07.486821 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:07.511538 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:07.570473 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74a030-bade-4da8-96a8-032e1c577bcb None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:07.616263 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.617576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.617866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.618144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/502c9194cb0348f185dd9a5262ad1d2e/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.621184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=502c9194cb0348f185dd9a5262ad1d2e, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.623409 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:07.624003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.632408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1462-36f1-4d45-85ce-c96ba0ffa6f1 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:07.661397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 371/767] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:49:07 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/502c9194cb0348f185dd9a5262ad1d2e/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:49:07.662405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:07.678262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.679721 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.680837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.681420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.682594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=226106a545904f6facb1dab0dec07656) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:07.683188 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.685070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.694036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 397/768] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:07 2022] GET /identity/v3/role_assignments?user.id=226106a545904f6facb1dab0dec07656&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:07.846840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1734c-ab59-4cac-8e6d-24e1da418e0c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:07.854772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:07.903137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:07.904016 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:07.904296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:07.904575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:07.905318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:07.906796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:07.928737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:07.930008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:07.930949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 372/769] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:49:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:07.936694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.075127 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1e6ed-9bf5-4d84-9723-51c911526684 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:08.111066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:08.111772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.112058 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.112333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.113086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:08.113486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:08.114975 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ed654-2a4f-452f-9974-d1792959b4fd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:08.120933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 398/770] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:49:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:08.126977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.253348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe26b-a71f-4da5-b574-a996c07f6ed4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:08.300242 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:08.300931 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.301214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.301528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.302264 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:08.302587 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:08.303932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:08.310083 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971a4e-4161-4cd4-9b31-3171d33ca516 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.310368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971a4e-4161-4cd4-9b31-3171d33ca516 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.310665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971a4e-4161-4cd4-9b31-3171d33ca516 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.312282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 399/771] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:08.312793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 373/772] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:08.317378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.317664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.317938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.323577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.335563 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:08.336754 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:08.355170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:08.382869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:08.420229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 400/773] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:08.446884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.540062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaba4ba-e3f4-4cc1-a5c6-6639a416bd37 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:08.573683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79a7b2-b59e-415e-939d-fe1fe970b356 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:08.609110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:08.609830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.610100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.610375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.611183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] RBAC: Adding query filter params (interface=public, service_id=45d23461fbb34b6f803faa0b456ca11c, region_id=RegionOne) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:08.611498 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:08.612891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cbe80-e0a7-46a6-a149-a1a0d2e12245 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:08.615914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9d654-b5aa-4b57-a940-16a40330b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:08.616937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9d654-b5aa-4b57-a940-16a40330b9ef None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.617424 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9d654-b5aa-4b57-a940-16a40330b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.618372 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9d654-b5aa-4b57-a940-16a40330b9ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.625491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 374/774] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/endpoints?service_id=45d23461fbb34b6f803faa0b456ca11c&interface=public®ion_id=RegionOne => generated 194 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:08.629698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843438c-d2b8-4528-8342-2c104f77a6fe None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.629698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843438c-d2b8-4528-8342-2c104f77a6fe None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.629698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843438c-d2b8-4528-8342-2c104f77a6fe None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.633089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 375/775] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:08.633888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843438c-d2b8-4528-8342-2c104f77a6fe None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.639175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9d654-b5aa-4b57-a940-16a40330b9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e3e9d654-b5aa-4b57-a940-16a40330b9ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:08.646443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:08.647462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:08.648374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:08.652635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 401/776] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:08.654087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.654377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.654658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.672276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:08.673501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:08.683855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:08.744946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 402/777] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:08.745866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.772921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:08.791150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843438c-d2b8-4528-8342-2c104f77a6fe None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:08.829586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:08.830403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.830595 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.830925 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.831556 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:08.833114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:08.845948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 376/778] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:08.847105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.852920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5862da-e863-48db-90c3-468eec31b166 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:08.900364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:08.901521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:08.901990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:08.902421 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:08.903500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:08.904035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:08.905765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:08.916221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 403/779] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:08.922417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:08.932976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:09.044415 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451cb0-f68e-4e0a-b384-ab270bf4d8b1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.086381 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.087219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.087498 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.087769 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.088715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.091627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:09.106172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:09.107332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:09.108294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 377/780] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:09.114022 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:09.148245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f13a0-5b17-49af-94d0-65ad05c6d3a6 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.154144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:09.212451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.213759 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.214241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.214828 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.238421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00090871-eec1-4372-ae19-59f5c5a4f9f7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.246372 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.248255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.278296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.279137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.285260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.285260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.285260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] RBAC: Adding query filter params (name=network) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:09.285260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.285260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.287716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 378/781] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:09.289842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:09.290991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:09.292221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:09.292221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:09.292221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:09.292221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:09.293290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:09.294396 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 404/782] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:09.299717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:09.393667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:09.407037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746bd3c-408a-4106-9063-17bdb2efad48 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.448316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.450230 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.450818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.451370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.452409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] RBAC: Adding query filter params (type=network) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:09.452788 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.454313 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.460782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 379/783] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:09.467029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:09.489623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:09.576204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880d647-0886-4916-bb93-cd0c956b9cfb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.612192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.612973 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.613356 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.613689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.614708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] RBAC: Adding query filter params (interface=public, service_id=ec73c52327814512b12b1d90b08c9a3b, region_id=RegionOne) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:09.615113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.616564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.622446 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 380/784] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:09 2022] GET /identity/v3/endpoints?service_id=ec73c52327814512b12b1d90b08c9a3b&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:09.631343 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:09.657355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:09.687280 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351fa98-3f14-42a2-b818-61646b720c67 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.743350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d12cb-26c2-4051-8eb9-0a7e0b3a74b4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:09.780392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.781204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.781595 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.781935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.782827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.784281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.790555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 381/785] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:09 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:09.828386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:09.829510 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:09.829992 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:09.830473 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:09.831500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:09.832001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:09.834221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:09.855028 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 405/786] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 556 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:09.861087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:09.977779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:09.994167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be633c38-9894-4b4f-ab78-bbfaa3f18db6 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:10.030809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:10.031641 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.031917 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.032190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.040796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:10.042360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:10.057373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:10.058403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:10.059529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 382/787] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:10.070465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:10.203182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e6632-b68e-4343-bbd0-85d4545a3843 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:10.226209 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:10.245724 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:10.246400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.246684 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.246999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.247837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:10.248144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:10.249628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dd201-5c77-4eed-8a0b-d275b34dcfb0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:10.269702 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 406/788] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:10.278613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:10.395993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cb52e-37c0-40fd-9f49-225cac30afbb None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.396349 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cb52e-37c0-40fd-9f49-225cac30afbb None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.396561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cb52e-37c0-40fd-9f49-225cac30afbb None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.398261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 407/789] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:10.401493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1e2f-08ca-490f-bcb2-033b93b4ddc6 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:10.404092 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.404614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.405097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.444141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:10.445379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:10.459220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:10.460048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.460329 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.460596 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.479866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:10.481502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:10.484761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:10.523875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:10.524896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:10.525912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:10.525912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:10.525912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:10.525912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:10.525912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 383/790] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:10.532532 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:10.571308 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 408/791] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:10.618824 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:10.639881 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edd56b-416d-4b78-8afe-a53bd71f41fa None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:10.707901 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:10.708586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.708909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.709190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.709940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:10.710239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:10.711615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9eb0-c7e4-4131-b732-a42b57749a50 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:10.741386 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 384/792] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:10.742902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982938e8-7e7e-4e43-a454-8597b65b9241 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.743186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982938e8-7e7e-4e43-a454-8597b65b9241 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.743593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982938e8-7e7e-4e43-a454-8597b65b9241 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.744835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 385/793] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:10.745439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982938e8-7e7e-4e43-a454-8597b65b9241 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:10.798398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c70b-d3ee-4819-ab91-b4d297336af1 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:10.807366 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:10.809718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:10.852332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f2fa8-3553-494f-93e2-c5c8332e895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:10.853153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f2fa8-3553-494f-93e2-c5c8332e895c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.853507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f2fa8-3553-494f-93e2-c5c8332e895c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.853909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f2fa8-3553-494f-93e2-c5c8332e895c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.866544 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982938e8-7e7e-4e43-a454-8597b65b9241 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:10.869095 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f2fa8-3553-494f-93e2-c5c8332e895c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-257f2fa8-3553-494f-93e2-c5c8332e895c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:10.880169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:10.881310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:10.882263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 409/794] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:10.883577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.883577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.883838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.907670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:10.909082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:10.909749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:10.910827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:10.911317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:10.911797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:10.920304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:10.922145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:10.937140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:10.938135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:10.939139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:10.941777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 386/795] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:10.942937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:10.993516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 410/796] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:10.994826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.073304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045d53-600c-4f88-af79-c685b0ce67b5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.090602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:11.110882 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:11.119153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09581e-b695-4547-aa73-04132fdb4020 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.120121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.121144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.121798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.122311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.123293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:11.123818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.125445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.135242 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 387/797] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:11.136419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.164913 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.165933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.166225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.166495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.167871 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:11.168278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.170253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.195251 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 411/798] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:11.196329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.273875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329cdb0-560a-4dad-aa41-315efd2a11c1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.319622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.321765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.322274 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.322989 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.328273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27153ae-0d0d-4a18-a5bf-30bf23f4cbe1 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.337378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:11.347715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:11.349510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:49:11.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:11.353372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 388/799] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:11.355527 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.374668 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.375486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.375766 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.376035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.387316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.388795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.400900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:11.409292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:11.410274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:11.411227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:11.411227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:11.411227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:11.411227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:11.412117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 412/800] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:11.413017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.424747 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:11.475847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32840b-a2e5-4d01-92ed-7a0a6d7c0f39 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.511220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b3b52-4785-48c4-9b42-cb4784dcd777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.512294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b3b52-4785-48c4-9b42-cb4784dcd777 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.512663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b3b52-4785-48c4-9b42-cb4784dcd777 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.512941 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b3b52-4785-48c4-9b42-cb4784dcd777 None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/7734eb173a1149839ce9c11ccc12013a/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.515090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b3b52-4785-48c4-9b42-cb4784dcd777 None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=7734eb173a1149839ce9c11ccc12013a, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.516758 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b3b52-4785-48c4-9b42-cb4784dcd777 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.521622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944884f7-e934-4f59-992f-0f09948e16eb None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.537405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 389/801] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:49:11 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/7734eb173a1149839ce9c11ccc12013a/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:49:11.537405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bae57-fc86-4fbe-938c-9e5a183d1739 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.537405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bae57-fc86-4fbe-938c-9e5a183d1739 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.537405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bae57-fc86-4fbe-938c-9e5a183d1739 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.537405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 390/802] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:11.537405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bae57-fc86-4fbe-938c-9e5a183d1739 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.573485 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.574251 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.574530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.574890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.575685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:11.575996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.577458 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdae9d-dec5-4461-8403-308d87e2fba7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.593790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 413/803] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:11.595999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.596470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.596922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.616279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:11.617761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:11.673741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:11.685986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bae57-fc86-4fbe-938c-9e5a183d1739 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.708752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:11.718125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 414/804] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:11.719292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.755320 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.756021 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.756300 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.756570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.757488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:11.757845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.761359 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.776176 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 391/805] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:11.777005 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.846705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30383508-7234-4444-a78c-7d228bcfffb8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.890804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.892092 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.892615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.893009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d9a20-73d0-4ac2-8ea7-47ba894b88cf None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:11.893356 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.905229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.907098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:11.926326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:11.927368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:11.928330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:11.928330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:11.928330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:11.928330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:11.930703 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 415/806] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:11.931852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.940566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:11.941456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:11.941737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:11.942008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:11.942694 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:11.944622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:11.961223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:11.962347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:11.963293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 392/807] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:11.964236 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:11.989698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.011898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.067621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83368-008d-4508-9615-482cccf836e7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.097194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe246d-8225-47cb-8e31-e8b73d564588 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.114518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.115636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.115916 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.116196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.125690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.127178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.136763 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.137571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.137856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.138138 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.138875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:12.139183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.140564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:12.149116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:12.150174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:12.151204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:12.158428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 393/808] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:12.158913 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 416/809] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:49:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:12.160609 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.160759 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.248374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.275186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b59fab-ab94-4245-8478-566167d0d06c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.312579 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108e16c-295a-4fb0-bc64-1b084d3ed7ed None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.313791 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.314579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.314966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.315343 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.316794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:12.316794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.318599 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.323646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 394/810] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:12.329160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.330593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.360501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.361903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.362648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.363076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.364353 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:12.365250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.367174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.393952 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 417/811] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:12.395328 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.442699 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb953351-0b4f-4677-b596-423ae5c46b04 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.484520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.485573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.485869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.486145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.495791 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.497928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.502402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6718c2-eab4-4c61-8949-8ffcaa8b3c74 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-4971b158-c8ce-40ac-8993-cc5fe855caf5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:12.517913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:12.518915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:12.519934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 395/812] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:12.522060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13b317-bb89-43ad-8df1-99207434b9f4 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.522377 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13b317-bb89-43ad-8df1-99207434b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.522765 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13b317-bb89-43ad-8df1-99207434b9f4 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.528560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 396/813] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:12.528560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13b317-bb89-43ad-8df1-99207434b9f4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.528560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.548943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.550003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.550497 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.550975 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.551925 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:12.552425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.554069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.560030 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 418/814] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:12.561040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.612595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.674393 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13b317-bb89-43ad-8df1-99207434b9f4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.690091 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecb51-214a-41e0-bed2-16fbef103686 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:12.715586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.716369 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.716723 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.717344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.738828 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:12.739533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.739829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.740094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.740967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:12.742542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.743889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848440b-a5bc-4df2-8e79-589b4d94edbb None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.761909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 419/815] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:12.763275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:12.763563 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:12.763840 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:12.782390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:12.783608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:12.799392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:12.801157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:12.837606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.860220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 420/816] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:12.862594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.896760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:12.921126 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 397/817] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:12.927183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:12.994149 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862b9b8-3b91-4319-ac20-7b82faf2c999 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.031370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.032046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.032327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.032599 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.033363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.033783 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadd22c-a9ee-4359-ad6c-fe1eee751ab7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.034747 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.035457 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.065266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 421/818] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:13.065266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.088811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.090118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.090465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.090738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.105829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.107764 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:13.133566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:13.134648 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:13.135618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:13.135618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:13.135618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:13.135618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:13.135870 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 398/819] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:13.137095 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.165222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.192815 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c4cc8-e0d9-42e4-beb3-a5835dba8f6d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.242816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.243734 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.244037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.244281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] PATH_INFO: `/v3/services/network` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.245306 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.247522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:13.265115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:13.266452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 14:49:13.267406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:13.272365 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 422/820] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:13.273404 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.275110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d550579-d25b-499c-8c4b-ed7080a93630 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.318396 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.319217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.319599 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.319868 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.333970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.335452 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:13.354066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:13.355107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:13.356199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:13.356199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:13.356199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:13.356199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:13.356199 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 399/821] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:13.356637 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.359874 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.404834 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989211b-3b85-422f-b958-0e2cf48f7fee None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.439639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.440259 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.440546 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.440813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.441593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:13.441898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.443283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.456029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 423/822] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:13.456957 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.460190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.469032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b842d26-e084-446a-887d-143394c5d99c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.504704 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.505517 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.505846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.506143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.506939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] RBAC: Adding query filter params (name=network) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:13.507249 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.508748 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.518767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 400/823] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:13.519761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.562715 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4270f34-a6fd-437e-86a9-11ec0b488f49 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.589993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.601067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.601798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.602081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.602354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.603086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:13.603493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.604995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.617760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 424/824] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:13.618788 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.628173 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26b39-417b-45da-aa64-e3a9a3602995 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.666046 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.667010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.667368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.667809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.676450 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.678119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:13.695447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:13.696485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:13.697513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:13.697989 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.699875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 401/825] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:13.700113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.739918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ead180-748b-4f6e-ab30-6b6551c13407 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.776528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.777354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.777645 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.777960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.778821 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] RBAC: Adding query filter params (type=network) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:13.779131 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.780522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.786056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 425/826] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:13.786863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.816114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05eebda-9312-4e22-81c4-3b55decce2c1 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.843871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.854313 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.855282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.855657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.856059 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.863976 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.865592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:13.881074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:13.882290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 402/827] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:13.883298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.897868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05e2bd-5de8-4d21-adf8-b8c1c6313e6d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:13.926573 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:13.939085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:13.940048 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.940335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.940609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.942089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:13.942417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:13.944443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebc711-b1ea-4078-b17a-2a283da34f65 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:13.960922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 426/828] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:13.962290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dc40b-c836-436a-a530-900e1561fa43 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:13.962535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dc40b-c836-436a-a530-900e1561fa43 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:13.962842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dc40b-c836-436a-a530-900e1561fa43 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:13.965409 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 427/829] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:13.965409 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dc40b-c836-436a-a530-900e1561fa43 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:13.995394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c0d25-097f-46d8-bcdd-b5c8b19f4924 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.032027 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.032862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.033260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.033656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.034531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:14.036015 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:14.057539 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 403/830] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:14.058469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:14.100848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dc40b-c836-436a-a530-900e1561fa43 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.109463 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:14.154296 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.154761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.155143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.155534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.156488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:14.156860 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:14.158580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ab60c-c7b5-4b3a-9d3d-04c58b7f5ed1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:14.177746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 428/831] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:14.179468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.179863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.180339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.199515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:14.201076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:14.204177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fda8e-617a-4c3c-958c-874e297e5b5a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.243669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:14.246180 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.247502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.248172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.248557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/226106a545904f6facb1dab0dec07656/roles/931260cf51714e41884b328407aed54f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.265372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=226106a545904f6facb1dab0dec07656, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:14.266990 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:14.286633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 404/833] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:49:14 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/226106a545904f6facb1dab0dec07656/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:49:14.286844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 429/833] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:14.287573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:14.313514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:14.438286 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:14.451872 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589943-b323-4845-8ff8-21de8bdee03d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.474355 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5416b0-0a13-467d-ab63-831a85e78f6e None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.501730 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.502971 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.503407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.503848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.505604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=502c9194cb0348f185dd9a5262ad1d2e) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:14.506358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:14.508673 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:14.517766 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 405/834] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:14 2022] GET /identity/v3/role_assignments?user.id=502c9194cb0348f185dd9a5262ad1d2e&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:14.517980 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.518630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.518872 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.519142 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.536225 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:14.544557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:14.545748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:14.546891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 430/835] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:14.552115 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:14.664377 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d01e6d-1008-4d25-8d47-2ede870f405e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.701168 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.701909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.702190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.702526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.703338 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:14.703646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:14.705052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:14.715058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 406/836] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:14.720649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:14.759018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:14.831624 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67f511-d45c-40d3-8ea6-a2336270be22 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:14.867470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:14.868290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:14.868574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:14.868842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:14.880263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:14.881774 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:14.900199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:14.901687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:14.902649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:14.902649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:14.902649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:14.902649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:14.905086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 431/837] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:14.913283 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:14.931741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.019862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493e972-9909-496b-a297-2203325d0e05 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.054792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.055483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.055761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.056033 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.056775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:15.057081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.058500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.071354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 407/838] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:15.076707 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.168094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.186207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229100e-9485-40cc-8791-92bd4c42b94b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.221768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.222606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.222884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.223164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] PATH_INFO: `/v3/users/swift` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.232261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.233820 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:15.250323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:15.251436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:15.253727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 432/839] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:15.258494 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.342379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.366105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48235680-03b2-4e66-a3a9-d356c1d378e3 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.402810 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.403523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.403802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.404069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.404921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:15.405225 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.406690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a34bf7-3c1e-4ca4-8983-9503a2068588 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.425086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 408/840] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:15.431744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.435610 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f41c9-75c5-45f1-8111-fc9e6cd05d27 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.435890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f41c9-75c5-45f1-8111-fc9e6cd05d27 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.436163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f41c9-75c5-45f1-8111-fc9e6cd05d27 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.437793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 409/841] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:15.442718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.443376 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.443376 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.464490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:15.466483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:15.539538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e291d-23ea-4b25-9c95-46f20f91c590 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.548138 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 410/842] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:15.571229 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.581843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.582675 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.583053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.583418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.599659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.659448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.667004 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.668533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.708095 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec1118-8af7-4cbe-aed5-a98e05d6a78f None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.754486 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.755341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.755620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.755885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.756538 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.757957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:15.770914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:15.772079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:15.772982 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:15.777444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 411/843] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:15.778373 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.785001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 433/844] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:15.790786 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.854001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.886145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7af1e4-401e-4650-8023-448a2f237d2c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.921960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853148-a75e-42e4-9d47-8795ef7e8b15 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:15.926935 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:15.935811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.937511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.938012 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.938728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.939878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:15.940371 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.942336 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.949452 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 412/845] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:15.955379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:15.962928 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:15.963740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:15.964017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:15.964279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:15.977064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:15.978630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:15.999255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:16.000248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:16.001658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:16.001658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:16.001658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:16.001658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:16.001658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 434/846] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:16.005949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:16.077320 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff131-6be3-4494-bf7b-0504ae97767c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:16.112897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:16.113523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.113798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.114069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.114779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:16.115076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:16.116553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:16.127867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 413/847] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:16.134200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:16.142306 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a04cd-f192-4d24-b6f5-7315d9e8784c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:16.182454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:16.183677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.184221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.184706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.185722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:16.186262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:16.187949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:16.202238 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 435/848] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:16.208318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:16.276734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:16.326065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd307a-9f88-4353-8623-e291e928e167 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:16.367739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:16.368534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.368816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.369083 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.376479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:16.377966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:16.392937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:16.393983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:16.394987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 414/849] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:16.401843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:16.507861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffc5eb-aace-47c5-a4dd-8df516cc8371 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:16.547110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:16.548185 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.548779 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.549398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.550417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:16.550948 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:16.552657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80c119-5644-4b06-bcb3-7514aeadaaa3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:16.567427 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 436/850] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:16.573852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:16.687360 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca887e-a445-4dbf-8cd2-8060cc7de2af None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:16.706145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:16.724651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:16.725371 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.725656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.725932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.726793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=7734eb173a1149839ce9c11ccc12013a) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:16.727108 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:16.728593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:16.731279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e83b7-88e9-479d-a594-86ad0e37541d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.731566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e83b7-88e9-479d-a594-86ad0e37541d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.731846 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e83b7-88e9-479d-a594-86ad0e37541d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.733658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 437/851] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:16.736144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 415/852] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:16 2022] GET /identity/v3/role_assignments?user.id=7734eb173a1149839ce9c11ccc12013a&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:16.739512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010646e-8b7b-46d4-8a11-8d73b7ab70f0 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.739804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010646e-8b7b-46d4-8a11-8d73b7ab70f0 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.740079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010646e-8b7b-46d4-8a11-8d73b7ab70f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.762079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-9010646e-8b7b-46d4-8a11-8d73b7ab70f0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:16.763405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-9010646e-8b7b-46d4-8a11-8d73b7ab70f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:16.778084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:16.845100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 438/853] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:16.868366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:16.944265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff3969-489e-4ed3-a2e4-78563fe3a45f None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.944814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff3969-489e-4ed3-a2e4-78563fe3a45f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.944814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff3969-489e-4ed3-a2e4-78563fe3a45f None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.947200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 439/854] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:16.955274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fde99-290d-44a3-b76f-de99b651798b None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:16.955608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fde99-290d-44a3-b76f-de99b651798b None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:16.955835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fde99-290d-44a3-b76f-de99b651798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:16.980374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-310fde99-290d-44a3-b76f-de99b651798b None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:16.981794 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-310fde99-290d-44a3-b76f-de99b651798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:16.995891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06a96-8188-4a89-8a41-d1874ffc3f5e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.055761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.057201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.057995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.058860 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.063806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 440/855] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:17.073161 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:17.081523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:17.082536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:49:17.083504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:17.086062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 416/856] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:49:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:17.086261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fde99-290d-44a3-b76f-de99b651798b None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.091629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.142579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:17.195513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:17.208513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481b46-2404-482b-91f6-d65686d4fcc5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.226612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fde99-290d-44a3-b76f-de99b651798b None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.246031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.246965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.247382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.247811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.248776 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:17.249185 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.250705 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.265526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 417/857] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:17.267184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.270736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.272623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.272921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.273199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.273956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.275400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:17.292694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:17.293888 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:17.295268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 441/858] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:17.301313 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.407220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437617ad-9eb5-4aa1-8b02-6abce0783f84 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.429658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:17.445660 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.446364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e3a1-5a5e-4239-83cf-f4e919813ff7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.446488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.446657 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.446913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.459689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.461781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:17.483934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:17.484999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:17.486034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:17.486034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:17.486034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:17.486034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:17.486034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 418/859] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:17.489168 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.489901 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.490193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.490464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.491191 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:17.491494 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.491687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.492882 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.499218 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 442/860] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:17.505294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.511887 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:17.597072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4115c11-1122-41a2-b867-949fa427249d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.630672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6984045-2162-4c39-b783-ae8704d314c4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.633224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.633943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.634226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.634501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.635239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:17.635550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.636940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.651159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 419/861] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:17.658873 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.670322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.671007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.671297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.671574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.672294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:17.672601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.674035 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.678912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 443/862] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:17.679188 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:17.775006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcdb1c-234c-4ac0-bf18-63045ff28c27 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:17.813668 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:17.814540 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:17.814823 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:17.815097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:17.823938 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:17.825569 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:17.844646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:17.845897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:17.847470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 420/863] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:17.855098 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:17.929419 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:17.964581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f43ed-2b7d-4f4d-ae22-eba38a070cca None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:18.000259 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:18.000975 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.001286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.001566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.002432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:18.002742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:18.004228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:18.024390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 444/864] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:49:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:18.034439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:18.119769 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:18.144951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d8f47-807e-4c34-889c-4d1722dfe643 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:18.180581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff77eb2-6f5e-47d0-826f-ba0ca6f86bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:18.181287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff77eb2-6f5e-47d0-826f-ba0ca6f86bd2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.181624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff77eb2-6f5e-47d0-826f-ba0ca6f86bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.181902 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff77eb2-6f5e-47d0-826f-ba0ca6f86bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.256075 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff77eb2-6f5e-47d0-826f-ba0ca6f86bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:18.257920 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff77eb2-6f5e-47d0-826f-ba0ca6f86bd2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:18.347560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:18.374849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 421/865] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:18.380743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:18.483683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e046cc-43e7-411f-a18b-d63afd2f983d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:18.498781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dac3f9-bf67-4f89-9283-ab1e9f21c959 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.499060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dac3f9-bf67-4f89-9283-ab1e9f21c959 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.499333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dac3f9-bf67-4f89-9283-ab1e9f21c959 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.501151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 422/866] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:18.506565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.506838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.507113 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.515231 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:18.526032 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:18.526747 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:18.527330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.527969 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:18.528296 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.528770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.543108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:18.545742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:18.570543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:18.571578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:18.572595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:18.572595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:18.572595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:18.572595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:18.574828 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 445/867] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:18.581781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:18.613463 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 423/868] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:18.636142 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:18.703096 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e95857-9107-4068-a814-2fd052ffb86d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:18.739318 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:18.739995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.740271 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.740540 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.741278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:18.741582 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:18.742950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:18.755341 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 446/869] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:18.761379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:18.772700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1ad5a-69a6-41d3-87d2-3505c19893ad None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:18.783159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:18.813877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:18.814554 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.814878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.815150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.815655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:18.817090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:18.818749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:18.845656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 424/870] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:18 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:49:18.897892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ff70c-8449-4f6c-b6b7-fb74b5090a08 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:18.951603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:18.952910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:18.953464 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:18.953947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:18.962650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:18.964581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-ad0a6c43-2e0d-4854-b6f9-ddb120486daf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:18.981451 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:18.982930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:18.983938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:18.986695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 447/871] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:18.995043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:19.107240 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2a2f3-1bb0-4bb3-ba04-de809b5e0196 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:19.134977 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5930412-0446-499b-8f78-97afdb30434b None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.135311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5930412-0446-499b-8f78-97afdb30434b None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.135504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5930412-0446-499b-8f78-97afdb30434b None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.137133 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 448/872] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:19.142637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2055953-f822-449f-8388-959043527bf8 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.142981 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2055953-f822-449f-8388-959043527bf8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.143258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2055953-f822-449f-8388-959043527bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.165355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:19.165355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.165355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.165355 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.165868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-b2055953-f822-449f-8388-959043527bf8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:19.165951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:19.165951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:19.166097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-b2055953-f822-449f-8388-959043527bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:19.166172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:19.177189 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 425/873] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:19.182888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:19.219559 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:19.241340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 449/874] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:19.267734 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2055953-f822-449f-8388-959043527bf8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:19.295364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:19.318252 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8926b9-090c-4b1b-a855-6739b0bc27ba None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:19.360024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:19.360890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.361281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.361682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.362619 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=226106a545904f6facb1dab0dec07656) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:19.363015 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:19.364557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:19.371655 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 426/875] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:19 2022] GET /identity/v3/role_assignments?user.id=226106a545904f6facb1dab0dec07656&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:19.381535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2055953-f822-449f-8388-959043527bf8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:19.423730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:19.425939 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.426448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.426942 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.427688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:19.429115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:19.443448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:19.445741 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:19.447557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 450/876] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:19.451520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:19.560783 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:19.571086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450355e-1f22-41ba-9bed-e9cc09efe9c7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:19.610436 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:19.611494 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.611986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.612485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.613489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:19.614466 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:19.616069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc917d1a-df0f-47ed-a468-2320d8c94459 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:19.621209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 427/877] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:19.629914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:19.733962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:19.747106 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae363d3-1939-48de-a833-7bb3ff615ba2 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:19.787068 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:19.787767 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.788050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.788322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.789057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:19.789393 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:19.790839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bad320-9cc0-48e5-8092-a3a7d921263f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:19.801403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 451/878] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:19.966786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c741810-2e03-49b3-b3fd-7df236a4f1dd None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.967169 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c741810-2e03-49b3-b3fd-7df236a4f1dd None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.967354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c741810-2e03-49b3-b3fd-7df236a4f1dd None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.969029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 428/879] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:19.976850 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31962-cfe7-4d2b-a478-c77e962be4c9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:19.977266 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31962-cfe7-4d2b-a478-c77e962be4c9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:19.977945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31962-cfe7-4d2b-a478-c77e962be4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:19.996851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-fdf31962-cfe7-4d2b-a478-c77e962be4c9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:19.998009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-fdf31962-cfe7-4d2b-a478-c77e962be4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:20.068908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 452/880] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:20.091797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c741810-2e03-49b3-b3fd-7df236a4f1dd None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:20.198157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c741810-2e03-49b3-b3fd-7df236a4f1dd None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:20.233640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fefc73-c6ee-4a75-a6c0-3f43ac1360be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:20.234325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fefc73-c6ee-4a75-a6c0-3f43ac1360be None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:20.234604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fefc73-c6ee-4a75-a6c0-3f43ac1360be None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:20.234879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fefc73-c6ee-4a75-a6c0-3f43ac1360be None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:20.235631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fefc73-c6ee-4a75-a6c0-3f43ac1360be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:20.237029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fefc73-c6ee-4a75-a6c0-3f43ac1360be None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:20.257500 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 429/881] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:20 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:21.224143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12c48e-c9b0-4e0d-9aa7-d071ddc3d787 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.224727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12c48e-c9b0-4e0d-9aa7-d071ddc3d787 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.224727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12c48e-c9b0-4e0d-9aa7-d071ddc3d787 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.226374 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 453/882] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:21.231970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.232414 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.232520 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.249839 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:21.251033 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:21.260334 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:21.326157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 430/883] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:21.349051 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12c48e-c9b0-4e0d-9aa7-d071ddc3d787 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:21.441402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:21.462570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12c48e-c9b0-4e0d-9aa7-d071ddc3d787 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:21.501000 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:21.501782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.502062 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.502333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.502995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:21.504383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:21.516679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:21.518207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:21.519207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 454/884] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:21.524731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:21.628695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1272f5-b987-4e76-91d3-e05cf34db01a None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:21.666599 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:21.667305 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.667584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.667860 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.668680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:21.668988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:21.670463 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:21.676496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 431/885] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:21.682414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:21.789575 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09af625-3530-4062-8c74-993dab6d2534 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:21.830833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:21.831514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.831802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.832078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.832796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:21.833112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:21.834531 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d9d0-acf0-4969-b24c-8a06d7cb031a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:21.840236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 455/886] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:21.846268 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:21.850550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:21.851656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdb354-d458-482d-814d-2c50c6800548 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.852367 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdb354-d458-482d-814d-2c50c6800548 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.852802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdb354-d458-482d-814d-2c50c6800548 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.854651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 456/887] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:21.859924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:21.860399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:21.860850 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:21.878473 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:21.879653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:21.883359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:21.963586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 457/888] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:21.984013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881486-24d4-4b6c-bab2-32a2b83b8f7a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:21.986555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.021702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.022672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.023129 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.023563 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.024521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] RBAC: Adding query filter params (interface=public, service_id=f04245ccd53b45faa243db6c7a1c315e, region_id=RegionOne) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:22.025044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.026501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.032239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 432/889] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:21 2022] GET /identity/v3/endpoints?service_id=f04245ccd53b45faa243db6c7a1c315e&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:22.040033 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.117430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0544a-a32a-4f02-9f05-ef1c35c232be None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.120785 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:22.154259 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94815369-b72e-4c2d-9907-5f4ea1dcdcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.155076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94815369-b72e-4c2d-9907-5f4ea1dcdcb6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.155354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94815369-b72e-4c2d-9907-5f4ea1dcdcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.155622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94815369-b72e-4c2d-9907-5f4ea1dcdcb6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.156715 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6626-3641-4b3d-814e-71bd5a891a14 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.163537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:22.167643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94815369-b72e-4c2d-9907-5f4ea1dcdcb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-94815369-b72e-4c2d-9907-5f4ea1dcdcb6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:22.176463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:22.177516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:22.178462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:22.182667 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 458/890] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:49:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:22.184007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e5730-1817-461d-ba12-e9546465dced None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.184303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e5730-1817-461d-ba12-e9546465dced None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.184568 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e5730-1817-461d-ba12-e9546465dced None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.185884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 459/891] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:22.186568 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e5730-1817-461d-ba12-e9546465dced None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.202041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d5aed-348f-4e72-9faa-3106607bcf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.203077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d5aed-348f-4e72-9faa-3106607bcf47 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.203590 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d5aed-348f-4e72-9faa-3106607bcf47 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.204065 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d5aed-348f-4e72-9faa-3106607bcf47 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.204956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d5aed-348f-4e72-9faa-3106607bcf47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.206593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d5aed-348f-4e72-9faa-3106607bcf47 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.221287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 433/892] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:49:22.221287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.221287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.221287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.240212 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:22.241535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:22.311898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e5730-1817-461d-ba12-e9546465dced None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.333724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 434/893] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:22.350274 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.350994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.351239 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.351505 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.351932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.352319 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:22.352621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.354078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.364406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 460/894] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:22.370169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.424890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:22.445820 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:22.477068 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77b5ad-7aa1-4117-8144-58724d2e326f None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.497295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e73cf1-acb8-4920-ab38-0fa1c3fe44a0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.530700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.531802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.532325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.532819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.533636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.534091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.534449 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.534731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.534998 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.536375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.546450 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.547912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.554441 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 435/895] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:22 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-99ea4b2a-13ab-4618-b8a5-5be6f794cb44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:22.574375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:22.575511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:22.576522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:22.576522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:22.576522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:22.576522 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:22.579243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 461/896] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:22.585970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.609134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13d02-a23b-4d59-956b-ab67b1e70c9d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.609905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13d02-a23b-4d59-956b-ab67b1e70c9d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.610275 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13d02-a23b-4d59-956b-ab67b1e70c9d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.612036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 462/897] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:22.617971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.618265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.618683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.645112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:22.646938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:22.720177 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707006-4f3f-4a64-9439-42634c47f774 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.729412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 463/898] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:22.745325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:22.746611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.766434 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.767126 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.767408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.767686 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.768429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:22.768739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.771409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.786297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 436/899] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:22.792929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.805918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:22.870923 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba850c-e117-4b57-9224-7427c886e3d8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.900718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92116925-9a5a-4123-b7f5-30beb59d9fbd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:22.917935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.919402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.919857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.920287 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.921107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.923123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.936096 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:22.937069 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:22.937517 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:22.937895 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:22.940061 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:22.941392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:22.942530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:22.945767 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 464/900] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:22.948687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:22.950479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:22.951717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:22.968182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:22.969297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:22.970743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 437/901] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:22.975942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.017581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.064488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6874d44-d70d-4839-a1a9-6066131022b5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.073962 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.080151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c20cb-406b-4ad1-9bfd-00f4f873e148 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.100789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.101526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.101813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.102086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.102808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:23.103115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.104517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.110104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 465/902] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:23.115546 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.122582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.123529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.124093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.124552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.125688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:23.126197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.127871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.149256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 438/903] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:49:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:23.154791 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.246127 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15306843-2d62-4f06-9875-9384efa58d2c None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.272669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.281324 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33590c25-df6c-47b3-a9bd-3b8bf18720cf None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.289743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.291157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.291674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.292160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.293529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:23.294108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.296394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.303781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 466/904] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:23.309535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.323189 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.324182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.324724 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.325221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.346976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.396368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.398478 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.418160 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710ce05-86c6-49e8-b850-f5e450d7f801 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.456320 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.457120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.457436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.457706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.458544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] RBAC: Adding query filter params (interface=public, service_id=95176915e92f43e3acc0357ac9ca05ad, region_id=RegionOne) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:23.459215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.460349 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.478417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 467/905] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/endpoints?service_id=95176915e92f43e3acc0357ac9ca05ad&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:23.479349 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.505323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.512822 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 439/906] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:23.518807 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.591614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.595348 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d74011-5944-4944-8a40-eaad9ca23e67 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.623062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f9d6c-72b9-47e8-9873-94f7d9fa0477 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.642883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.643782 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.644208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.644621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.645849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.647493 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.661393 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 468/907] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:49:23.665219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.666134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.666418 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.666687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.679543 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.681057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:23.701672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:23.702741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:23.703695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:23.703695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:23.703695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:23.703695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:23.703695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 440/908] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:23.710134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.822903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8bb73-0d69-4eb5-bfdb-6fc7cb018d8a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:23.861078 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:23.861807 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:23.862088 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:23.862360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:23.863169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:23.863479 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:23.864886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:23.878849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 469/909] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:23.884437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:23.939116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:23.992646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ddb76-0f67-4e91-b295-337ca0516ffe None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:24.027914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:24.028735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.029014 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.029309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.036420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:24.037915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:24.051965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:24.052962 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:24.054073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 441/910] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:24.063259 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:24.089706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:24.176420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5599aab-e24a-4aa5-8ca4-67b712156ca1 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:24.211291 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:24.211889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.212174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.212453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.213284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:24.213589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:24.215044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cd858-9e55-4449-ac57-f4e5c1c829cf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:24.228245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 470/911] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:24.234379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:24.338870 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287a98-92db-4df7-8663-025a698e7247 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:24.349206 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:24.374361 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:24.375032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.375314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.375585 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.376426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] RBAC: Adding query filter params (role.id=ef61c989c0eb46348b817f588d6116be, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=226106a545904f6facb1dab0dec07656) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:24.376743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:24.378237 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8f5e6-4a07-4257-9221-6ccec8ebd71e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:24.385257 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 442/912] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:24 2022] GET /identity/v3/role_assignments?user.id=226106a545904f6facb1dab0dec07656&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=ef61c989c0eb46348b817f588d6116be => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:24.588185 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7fb2-df54-403f-97be-c8e6d57ab83a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.588719 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7fb2-df54-403f-97be-c8e6d57ab83a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.588719 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7fb2-df54-403f-97be-c8e6d57ab83a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.590390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 471/913] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:24.595856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.596148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.596429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.614934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:24.616202 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:24.696580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 443/914] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:24.719989 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7fb2-df54-403f-97be-c8e6d57ab83a None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:24.811701 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:24.829000 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7fb2-df54-403f-97be-c8e6d57ab83a None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:24.870165 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:24.870991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.871302 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.871623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.872295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:24.873723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6acfb217-5b24-41c5-b509-0d63044bb141 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:24.886221 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:24.887338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:24.888343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 472/915] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:24.893980 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:24.918278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e0fed-92d9-403e-babb-6303fb3a0fda None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.918642 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e0fed-92d9-403e-babb-6303fb3a0fda None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.918855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e0fed-92d9-403e-babb-6303fb3a0fda None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.920664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 473/916] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:24.926676 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:24.927026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:24.927245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:24.944706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:24.945945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:25.021586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5238147-37b3-4e88-8537-4c672bb6fb36 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.023728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:25.027311 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 474/917] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:25.050028 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.064080 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.064775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.065056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.065390 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.066443 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:25.066760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.068591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.075425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 444/918] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:25.081020 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.146748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:25.174326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c394cf-01f8-4a63-8986-b0e26736d07a None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.200575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f691265-3763-4587-a4cc-e5d85d208e5c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.211181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.216264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.216846 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.217397 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.218316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.219978 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:25.233276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:25.234414 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:25.235395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:25.238243 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 475/919] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:25.244360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.246548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.247243 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.247529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.247807 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.248731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:25.248957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.250827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.260983 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 445/920] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 14:49:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:25.262143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.303804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:25.379350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553ceba-d84d-46de-904d-e75be8e57f3e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.394524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94add8-a284-4180-9f2e-08750c901f16 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.414669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.415281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.415565 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.416055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.416557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:25.416869 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.418354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.428224 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 476/921] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:25.429104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.435610 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.436603 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.437105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.437719 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.438714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.439359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:25.441547 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.473716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 446/922] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:49:25.554826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80572785-0a5d-4034-97fe-7fa68f0cbcb0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.600675 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.601399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.601655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.601931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.602646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:25.602954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.604494 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.610865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 477/923] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:25.617160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.729060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:25.748014 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561978e-88c6-40ff-8fbf-a5a101887fb8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.784112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.784793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.785092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.785390 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.786194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] RBAC: Adding query filter params (interface=public, service_id=c84f72d1f877433ea8d2bd6fd4b2efc6, region_id=RegionOne) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:25.786508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.787903 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a3bc2-5e3d-4f3c-89f8-402cbe625ef1 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.793710 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 447/924] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:25 2022] GET /identity/v3/endpoints?service_id=c84f72d1f877433ea8d2bd6fd4b2efc6&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:25.799108 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:25.873077 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:25.913433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2eaa6-9f7a-4e91-a83b-d3cd509614e8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:25.948561 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c49c-eb7f-42bb-a02b-71d8b937c268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:25.949687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c49c-eb7f-42bb-a02b-71d8b937c268 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:25.950188 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c49c-eb7f-42bb-a02b-71d8b937c268 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:25.950648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c49c-eb7f-42bb-a02b-71d8b937c268 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:25.951528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c49c-eb7f-42bb-a02b-71d8b937c268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:25.953100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c49c-eb7f-42bb-a02b-71d8b937c268 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:25.959994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 478/925] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:25 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:49:26.061071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d401bad-b58a-41ca-9c32-3dc1e02102bc None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.061337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d401bad-b58a-41ca-9c32-3dc1e02102bc None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.061563 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d401bad-b58a-41ca-9c32-3dc1e02102bc None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.063379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 448/926] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:26.069398 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.070015 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.070539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.089523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:26.091203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:26.167394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 479/927] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:26.190172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d401bad-b58a-41ca-9c32-3dc1e02102bc None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:26.303900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d401bad-b58a-41ca-9c32-3dc1e02102bc None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:26.340409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:26.341194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.341503 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.341772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.342437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:26.343866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:26.356991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:26.358067 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:26.358992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 449/928] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 14:49:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:26.366667 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:26.430262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:26.475849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb519e-e4da-44ab-a934-c14fdbaef6b6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:26.516190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:26.516894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.517169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.517481 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.518249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:26.518558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:26.519993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c6f42-2094-4e51-ac5c-a2413fdd4e6b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:26.526581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 480/929] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:26.532129 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:26.550169 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:26.656737 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173747e7-84b3-44cc-af1d-0a34f40da549 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:26.692166 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:26.692842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.693118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.693419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.695307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a62a-7226-41b0-9d6a-e98d20a3adad None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.695775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a62a-7226-41b0-9d6a-e98d20a3adad None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.695870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a62a-7226-41b0-9d6a-e98d20a3adad None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.696596 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:26.697131 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:26.697462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 481/930] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:26.699019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:26.705591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 450/931] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:26.705848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c4361-96df-47b9-a705-42e1380a074e None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.706056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c4361-96df-47b9-a705-42e1380a074e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.706448 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c4361-96df-47b9-a705-42e1380a074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.710942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:26.726278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-399c4361-96df-47b9-a705-42e1380a074e None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:26.728634 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-399c4361-96df-47b9-a705-42e1380a074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:26.802831 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:26.825780 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 482/932] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:26.827744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcd4c3-c218-4833-a3e5-31615617019f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:26.847994 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c4361-96df-47b9-a705-42e1380a074e None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:26.876140 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:26.877129 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:26.877451 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:26.877727 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:26.878914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:26.880560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:26.909425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 451/933] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:26.934777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:26.975295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c4361-96df-47b9-a705-42e1380a074e None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:27.026284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:27.027128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.027405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.027739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.040631 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:27.050512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:27.051610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:27.052532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 483/934] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:49:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:27.058344 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:27.170051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e395d6-3e36-40ae-a0d2-f4584f323814 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:27.206632 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:27.207792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.208333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.208814 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.209842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:27.210372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:27.211977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:27.213779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:27.224296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 452/935] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:49:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:27.229912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:27.337430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070b805-47de-430a-81d1-e6725472caf0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:27.348320 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:27.377286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:27.378215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.378501 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.378775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.394432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:27.396289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:27.419581 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:27.420673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:27.421666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:27.421666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:27.421666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:27.421666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:27.421666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 484/936] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:27.426994 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:27.533314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8f92d-455e-4721-8567-06ee2b65d1e8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:27.568736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:27.569441 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.569725 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.570009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.570746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:27.571099 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:27.572531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:27.584623 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 453/937] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:27.590162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:27.691858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b8b1f-9493-40e2-95b4-4c00dd23b994 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:27.726187 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:27.727043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.727321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.727591 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.736857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:27.738388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:27.755069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:27.756189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:27.757219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:27.761369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 485/938] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:49:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:27.762880 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:27.820395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:27.870152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a545c-3977-4cc9-bbf7-1e6f22d04143 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:27.905549 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:27.906526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.906991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.907419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.908408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:27.908872 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:27.910489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3b804-6dbc-4ed0-adac-3b2e16a63a3f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:27.929294 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 454/939] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:49:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:27.934775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:27.936909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15562ba-1da8-4c98-a6fc-dd3cce50fdf8 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.937194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15562ba-1da8-4c98-a6fc-dd3cce50fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.937498 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15562ba-1da8-4c98-a6fc-dd3cce50fdf8 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.938956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 455/940] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:27.945006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:27.945598 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:27.946087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:27.967806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:27.969683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:27.981711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.038343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682066cd-a25d-483d-a7a6-0eda4b6af0cd None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.063524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 456/941] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:28.079817 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.080629 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.080938 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.081178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.089132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.092402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.093890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:28.111690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:28.112698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:28.113739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:28.113739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:28.113739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:28.113739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:28.117607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 486/942] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:28.118918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.152330 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.202548 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.232322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048df0c8-0eb3-4650-9923-5463df5e0ebd None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.235735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddd986-6b39-4a7c-9ade-3f750851732c None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.269567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.270250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.270902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.271175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.271907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:28.272216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.273639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.284912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.285940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.286431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.287024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.288032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.289960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.291245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 487/943] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:28.292146 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: WARNING keystone.server.flask.common [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:49:28.292700 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.321121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.sql.core [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:49:28.321121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:49:28.321121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [parameters: {'id': 'a0a842e7b96c4c12a425da266786ed65', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:49:28.321121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=213542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-f393e01c-5043-489d-9c75-64ff7377f85f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a0a842e7b96c4c12a425da266786ed65.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a0a842e7b96c4c12a425da266786ed65. May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application conn.query(q) May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:49:28.338580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application result.read() May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 14:49:28.339624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application next(self.gen) May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application yield resource May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application next(self.gen) May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise self.value May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application session.commit() May 19 14:49:28.340746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self.session.flush() May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application compat.raise_( May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:49:28.342037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:49:28.343052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application conn.query(q) May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application result.read() May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:49:28.343929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [parameters: {'id': 'a0a842e7b96c4c12a425da266786ed65', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.344876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:28.345798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a0a842e7b96c4c12a425da266786ed65. May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 457/944] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 14:49:28 2022] POST /identity/v3/projects => generated 262 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795745e-70bf-40da-bd0e-bfd2110b52cd None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.353492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795745e-70bf-40da-bd0e-bfd2110b52cd None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.355548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795745e-70bf-40da-bd0e-bfd2110b52cd None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.356679 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 458/945] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:28 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:28.357501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795745e-70bf-40da-bd0e-bfd2110b52cd None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.406268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fe199-0c84-4eeb-a915-1e3abb251eba None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.442898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.443708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.443995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.444269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.446763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.453996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.455651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.458467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.463015 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795745e-70bf-40da-bd0e-bfd2110b52cd None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-23370b2c-422d-4ce2-a882-be12ef0928ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:28.471182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:28.472376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 488/946] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:28.473706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.474372 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.474372 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.491512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:28.492867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:28.504553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.505400 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.505681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.505952 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.513034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.514651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:28.529081 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:28.530098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 459/947] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:28.531072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.563505 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 489/948] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:28.564343 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.654399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ee88-9d38-4042-b3d4-0914af42b519 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.672909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad16bda-4284-4174-a352-f8462be77220 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.690428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.691111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.691394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.691675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.692474 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:28.692784 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.694259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.698854 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.708023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 460/949] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:28.708789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.716742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.717453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.717765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.718012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.718894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:28.719213 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.720658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.732766 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:28.749270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 490/950] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:28.749270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.839082 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82925a15-02b1-46b7-8549-02a35ac9ec1b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.885979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674e23b-4670-41e9-b7c3-df987c9bd405 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:28.887125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.888263 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.888551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.888827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.890534 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.891981 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:28.908970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:28.910444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:28.911667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:28.916255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 461/951] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:28.917582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:28.933557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:28.935758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:28.935758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:28.935758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/226106a545904f6facb1dab0dec07656/roles/ef61c989c0eb46348b817f588d6116be` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:28.939082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=226106a545904f6facb1dab0dec07656, role_id=ef61c989c0eb46348b817f588d6116be)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:28.941433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:28.959820 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 491/952] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:49:28 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/226106a545904f6facb1dab0dec07656/roles/ef61c989c0eb46348b817f588d6116be => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:49:29.041323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:29.055722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f809d19-704e-466f-8571-27f4bca16f49 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:29.097269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:29.097995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.098279 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.098557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.099295 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:29.099659 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:29.101152 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:29.107757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 462/953] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:29.113329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:29.167254 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:29.235135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e697e-4f73-43ac-a5df-7f8d2be9227d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:29.271143 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:29.271833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.272117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.272388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.273107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:29.273442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:29.274832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20c969-7352-473b-8829-46dfcaa22902 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:29.280622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 492/954] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:29.286409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:29.389554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb0c5f-1a53-4e5a-831e-31a0c8860095 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.389841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb0c5f-1a53-4e5a-831e-31a0c8860095 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.390121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb0c5f-1a53-4e5a-831e-31a0c8860095 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.391518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 493/955] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:29.397310 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.397601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.397798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49e80-e31c-4a10-942f-a1fc9da5c539 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:29.397917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.418588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:29.420175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:29.440019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235004ba-3559-406f-aede-d8560499108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:29.441086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235004ba-3559-406f-aede-d8560499108b None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.441697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235004ba-3559-406f-aede-d8560499108b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.442175 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235004ba-3559-406f-aede-d8560499108b None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.443155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235004ba-3559-406f-aede-d8560499108b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:29.444741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235004ba-3559-406f-aede-d8560499108b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:29.470039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 463/956] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:29.514755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 494/957] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:29.539296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235004ba-3559-406f-aede-d8560499108b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:29.542550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:29.664159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235004ba-3559-406f-aede-d8560499108b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:29.701865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:29.702683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.702963 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.703291 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] PATH_INFO: `/v3/services/compute` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.703953 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:29.705398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:29.717613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:29.718741 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:29.719692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 464/958] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:29.724901 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:29.760716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:29.830216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb71527-e31e-4aa2-aec3-063a4319f293 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:29.864638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:29.865342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:29.865620 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:29.865886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:29.866593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:29.866894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:29.868289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6274e-a570-4dd3-9544-6b8aae4e3eb6 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:29.873744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 495/959] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:29.879038 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:29.947767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:29.984612 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f911e-32a7-448a-8fdd-39363e6e353a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:30.019988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:30.021035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:30.021342 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:30.021621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:30.022341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] RBAC: Adding query filter params (type=compute) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:30.022651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:30.024049 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ade97-b0d0-42a2-baef-f6576ab7e869 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:30.029375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 465/960] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:31.172193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cf1dd-05a4-4d3e-8833-5d8541773a95 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.173090 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cf1dd-05a4-4d3e-8833-5d8541773a95 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.173543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cf1dd-05a4-4d3e-8833-5d8541773a95 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.175323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 496/961] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:31.181185 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b04ab-5c6f-470d-8c10-d981b93db7b6 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.182116 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b04ab-5c6f-470d-8c10-d981b93db7b6 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.182614 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b04ab-5c6f-470d-8c10-d981b93db7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.200778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f55b04ab-5c6f-470d-8c10-d981b93db7b6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:31.202681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f55b04ab-5c6f-470d-8c10-d981b93db7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:31.274431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 466/962] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:31.299796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cf1dd-05a4-4d3e-8833-5d8541773a95 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:31.354825 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d1c0d-2072-4023-911a-8faacd38c762 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.355163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d1c0d-2072-4023-911a-8faacd38c762 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.355389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d1c0d-2072-4023-911a-8faacd38c762 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.357217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 467/963] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:31.362214 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.362499 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.362780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.381077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:31.382453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:31.400979 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:31.406423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cf1dd-05a4-4d3e-8833-5d8541773a95 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:31.444176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:31.444933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.445309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.445657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.446489 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:31.448162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:31.449702 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: WARNING keystone.server.flask.common [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 14:49:31.451421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:31.463476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 468/964] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:31.466413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.sql.core [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:49:31.466413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:49:31.466413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [parameters: {'user_id': '17de16e14e1d4a12af33e148666080d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:49:31.466413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=213541) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application conn.query(q) May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result.read() May 19 14:49:31.475024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:49:31.476081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application next(self.gen) May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application yield resource May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application next(self.gen) May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.session.rollback() May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.force_reraise() May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise self.value May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application session.commit() May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 14:49:31.477152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._prepare_impl() May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.session.flush() May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._flush(objects) May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application compat.raise_( May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application flush_context.execute() May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rec.execute(self) May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 14:49:31.478170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 14:49:31.479141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result = self._query(query) May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application conn.query(q) May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application result.read() May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 14:49:31.480076 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [parameters: {'user_id': '17de16e14e1d4a12af33e148666080d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:31.481046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 14:49:31.483902 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:31.486228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 497/965] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 14:49:31 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:31.488726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:31.492456 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:31.627384 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a051-d141-4e52-835f-b9fbeae00963 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:31.632049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faf3-374f-41c4-8dfb-f73eee843788 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:31.673575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:31.674835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.675431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.675857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.676776 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e2089-c729-443c-a16c-4444be086546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:31.677789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2089-c729-443c-a16c-4444be086546 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.678189 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2089-c729-443c-a16c-4444be086546 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.678809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2089-c729-443c-a16c-4444be086546 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.685145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:31.687025 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:31.692067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e2089-c729-443c-a16c-4444be086546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-558e2089-c729-443c-a16c-4444be086546 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:31.701870 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:31.702991 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:31.703899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:31.706640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:31.707647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 14:49:31.709170 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:31.709851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 498/966] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:49:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:31.709851 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e2089-c729-443c-a16c-4444be086546 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:31.711938 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 469/967] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:31.721308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:31.747439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:31.782605 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:31.835543 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bcf0e-f575-4d9a-bb44-efaaae3c31ca None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:31.850693 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e2089-c729-443c-a16c-4444be086546 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:31.873619 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:31.874674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.875226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.875604 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.876404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:31.876735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:31.878186 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add4a62-e801-461a-aa28-41ff29587c39 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:31.886462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:31.887219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.887499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.887773 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.888518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:31.888822 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:31.890434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994774ac-056f-49ae-8c1e-adf3586aff03 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:31.906593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 499/968] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:49:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:31.906593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab53512-364b-4607-be64-d2edade450db None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.906593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab53512-364b-4607-be64-d2edade450db None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.906593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab53512-364b-4607-be64-d2edade450db None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.906593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 500/969] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:31.909154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab53512-364b-4607-be64-d2edade450db None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:31.921019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 470/970] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 14:49:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:31.922580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:31.922867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:31.923142 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:31.941998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:31.943252 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:32.037891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 471/971] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:32.039753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.052606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.062796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab53512-364b-4607-be64-d2edade450db None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.102751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.114723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.115567 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.115854 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.116130 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.135072 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.137073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:32.160958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:32.162041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:32.163050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:32.163050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:32.163050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:32.163050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:32.163050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 501/972] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:32.168152 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.183216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef7092-17e9-4e62-bc05-fd2ea8f4de2b None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.235535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.236372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.236653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.236926 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.237635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.239029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-271c7c2c-50c8-4005-84de-83a062e30a9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:32.254508 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:32.255731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:32.256651 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:32.259467 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 472/973] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:32.261623 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f2b24-473c-481e-bcd1-d4d89f3a84af None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.261927 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f2b24-473c-481e-bcd1-d4d89f3a84af None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.262204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f2b24-473c-481e-bcd1-d4d89f3a84af None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.263570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 473/974] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:32.264984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f2b24-473c-481e-bcd1-d4d89f3a84af None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.291926 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed501f0-a7d7-4984-8e6b-f5183b2a8205 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.326708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.327408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.327693 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.327973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.328706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:32.329010 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.330445 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df9e8-120f-4120-afb0-a373ea89032d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.331967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.347530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 502/975] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:32.347530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.347530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.347530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.361985 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:32.363542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:32.374959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f2b24-473c-481e-bcd1-d4d89f3a84af None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.377487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.411609 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.412282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.412557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.412855 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.413601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:32.413908 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.415289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.423814 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 474/976] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:32.426164 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.438789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 503/977] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:32.439752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.535942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987eee8-9015-4268-ab97-b85eeebe6419 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.541921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb966a-f0a8-414d-ac7e-d688e666e2f0 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.564889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.570210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.571047 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.571327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.571621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.576485 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.577157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.577466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.577736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.578442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:32.578743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.579203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.580107 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.580646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.585364 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 504/978] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:32.586015 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:32.595282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:32.596528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 475/979] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:32.597602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.627853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.693482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a058f-300e-4e96-aa13-e5a0f6a3489a None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.699203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443702da-9349-46c1-91af-a2c99ef60d6e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.730997 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.731671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.731952 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.732222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.732939 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.733838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.734369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.734523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.734796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.735061 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.735855 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] RBAC: Adding query filter params (interface=admin, service_id=c84f72d1f877433ea8d2bd6fd4b2efc6, region_id=RegionOne) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:32.736162 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.737529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.743529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 476/980] 10.222.0.73 () {56 vars in 1249 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/endpoints?service_id=c84f72d1f877433ea8d2bd6fd4b2efc6&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:32.744390 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.750181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 505/981] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:32 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:32.750712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.817169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.854244 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:32.864846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074d00d-7a53-4c58-88e4-30d540cbb8b8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.884770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f690249-ac53-447a-b429-91e686793a27 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:32.904204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.905309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.905798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.906275 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.907343 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:32.907908 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.909470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.923549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:32.924245 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:32.924528 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:32.924806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:32.925559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:32.926983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:32.930545 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 477/982] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:32.936414 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:32.939253 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 506/983] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/services => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:49:33.051947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b55c55-08fb-4920-a23c-597de8e033c3 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:33.088871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:33.089487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:33.089762 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:33.090112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:33.111097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:33.166513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:33.168220 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:33.269486 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 478/984] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:32 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:49:33.275571 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:33.383440 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e08782-aff2-4330-aa7f-d25e65db5432 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:33.393437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:33.420588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:33.421445 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:33.421728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:33.421997 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:33.433576 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:33.435081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:33.452534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:33.453873 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:33.459674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:33.459674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:33.459674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:33.459674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:33.459674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 507/985] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:33.461543 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:33.519159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:33.576452 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc556e7-4602-4326-afdf-92666e78d489 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:33.611917 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:33.612590 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:33.612863 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:33.613128 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:33.613884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:33.614185 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:33.615555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:33.635174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 479/986] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:33.636690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:33.746014 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c480d-1f03-466b-b284-46012f583ebf None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:33.781454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:33.782416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:33.782813 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:33.783218 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:33.790257 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:33.791823 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:33.806358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:33.807507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:33.808716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:33.810405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 508/987] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:33.816137 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:33.920226 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7e140-293b-45ae-bad9-75752e214952 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:33.932111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:33.960074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:33.960846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:33.961201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:33.961612 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:33.962479 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:33.962858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:33.964368 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db4e5-e7d6-47a1-aebf-5f7f618401c2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:33.977310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 480/988] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:33.983282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:34.006900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:34.092286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f8af2-ad1b-47b4-a03e-eb8dc268abbc None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:34.127623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:34.128437 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:34.128814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:34.129182 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:34.130144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] RBAC: Adding query filter params (role.id=ef61c989c0eb46348b817f588d6116be, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=226106a545904f6facb1dab0dec07656) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:34.130456 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:34.131889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8a507-c6f4-46c0-a5f7-f732a1763b1a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:34.145730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 509/989] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:33 2022] GET /identity/v3/role_assignments?user.id=226106a545904f6facb1dab0dec07656&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=ef61c989c0eb46348b817f588d6116be => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:35.051142 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275e776-94c0-4f29-bc21-c061f8c27535 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.051673 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275e776-94c0-4f29-bc21-c061f8c27535 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.051673 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275e776-94c0-4f29-bc21-c061f8c27535 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.052930 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 481/990] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:35.058639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.059156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.059156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.077709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:35.078913 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:35.148789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243b7a3-664b-445b-8807-4a367ae9f124 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.149162 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243b7a3-664b-445b-8807-4a367ae9f124 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.149387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243b7a3-664b-445b-8807-4a367ae9f124 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.151076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 482/991] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:35.152477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 510/992] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:35.157854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.158139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.158410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.174627 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.175312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:35.176494 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:35.191538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.249563 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 483/993] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:35.272316 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.284406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354d63c-6679-462c-92a4-ba8243a7f405 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.321051 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.321837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.322104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.322380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.323043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.324662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:35.337755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:35.338829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:35.339753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 511/994] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:35.344981 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.387011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.389836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec54fc1-d246-4c19-b209-9d78f0d8bb86 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.426371 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.427194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.427472 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.427744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.428404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.430327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.432683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:35.446700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:35.448013 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 14:49:35.449120 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:35.451697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 484/995] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:35.457128 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b887d-06d9-4687-9334-d14b5acc7ef8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.458200 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.495764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.496596 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.496949 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.497329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.498116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:35.498491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.499952 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.505190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 512/996] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:35.510751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.569430 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077eb61-0f8a-4beb-9bed-b3b580fc6dab None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.608647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.609466 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.609745 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.610135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.610885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:35.611161 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.612679 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.618442 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 485/997] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:35.624281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.625193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44394a02-11d3-46a7-bd26-9f0ba40c18b9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.660716 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.671621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.672308 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.672600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.672867 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.674024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] RBAC: Adding query filter params (type=compute) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:35.674335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.676094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.681659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 513/998] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:35.687417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.696296 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.744426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb59e7-c563-4d7c-94a0-f6181a2abad4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.781736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.782417 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.782701 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.782976 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.783694 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:35.784002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.785433 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.790695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 486/999] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:35.793928 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa767d2-e2da-4a65-a4e7-8aea3c489bea None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.796589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.830276 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.830935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.831224 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.831502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] PATH_INFO: `/v3/endpoints` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.832293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] RBAC: Adding query filter params (interface=public, service_id=83e77ed1067f4ec7ac9cfa1e5696e8a4, region_id=RegionOne) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:35.832621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.834020 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.839509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 514/1000] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/endpoints?service_id=83e77ed1067f4ec7ac9cfa1e5696e8a4&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:35.844898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:35.895865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.907001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c77772-7376-42a5-ae59-89b48679ed23 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.915663 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:35.943239 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d66b8-3dd5-4e1b-826e-d5c4884ea204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:35.943975 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d66b8-3dd5-4e1b-826e-d5c4884ea204 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:35.944261 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d66b8-3dd5-4e1b-826e-d5c4884ea204 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:35.944538 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d66b8-3dd5-4e1b-826e-d5c4884ea204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:35.945317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d66b8-3dd5-4e1b-826e-d5c4884ea204 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:35.946707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d66b8-3dd5-4e1b-826e-d5c4884ea204 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:35.948781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7f0c-7bec-4714-b1c7-fa64ad7c496b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:35.964788 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 487/1001] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:49:36.003239 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b46cdc-981b-4629-8e85-0d049cd3e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:36.003954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b46cdc-981b-4629-8e85-0d049cd3e02e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:36.004240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b46cdc-981b-4629-8e85-0d049cd3e02e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:36.004518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b46cdc-981b-4629-8e85-0d049cd3e02e None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:36.005215 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b46cdc-981b-4629-8e85-0d049cd3e02e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:36.006673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b46cdc-981b-4629-8e85-0d049cd3e02e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:36.014043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 515/1002] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:35 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:49:36.443769 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d48c-f3b0-4b3f-ad86-32d8f247f350 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:36.444322 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d48c-f3b0-4b3f-ad86-32d8f247f350 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:36.444322 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d48c-f3b0-4b3f-ad86-32d8f247f350 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:36.445821 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 488/1003] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:36.451396 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:36.451827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:36.451908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:36.469770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:36.470990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:36.543339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 516/1004] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:36.565563 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d48c-f3b0-4b3f-ad86-32d8f247f350 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:36.678267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d48c-f3b0-4b3f-ad86-32d8f247f350 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:36.713321 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:36.714051 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:36.714326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:36.714590 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] PATH_INFO: `/v3/services/image` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:36.715309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:36.716714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:36.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:36.734960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:36.735900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 489/1005] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:49:36 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:36.736809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:36.798224 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:36.840691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27cfe0-10db-440a-b87a-a28206472844 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:36.875125 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:36.875799 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:36.876073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:36.876340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:36.877041 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:36.877389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:36.878765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728889ba-be37-4f5d-81aa-5f0d98fa36f1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:36.884169 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 517/1006] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:36.889860 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:36.909999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:36.996613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26665b21-f82f-4746-b0fe-e6e1b0d09bca None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:37.031251 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:37.031919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:37.032200 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:37.032470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:37.033184 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] RBAC: Adding query filter params (type=image) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:37.033519 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:37.034905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1379c-2f89-4473-85c1-ab73a797ecde None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:37.045277 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 490/1007] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:38.255829 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467d4c2-cd02-491f-ad9d-71351570fc96 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.256380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467d4c2-cd02-491f-ad9d-71351570fc96 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.256380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467d4c2-cd02-491f-ad9d-71351570fc96 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.257866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 518/1008] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:38.263624 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.264063 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.264162 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.281365 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdfe40-052c-4fd4-a37b-1121e8b4241d None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.281604 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdfe40-052c-4fd4-a37b-1121e8b4241d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.281910 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdfe40-052c-4fd4-a37b-1121e8b4241d None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.282122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:38.283473 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 519/1009] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:38.283601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:38.288355 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.288601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.288903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.308178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:38.309529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:38.358509 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 491/1010] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:38.378695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 520/1011] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:38.382266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:38.394124 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:38.401337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:38.459509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:38.486602 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6eb6b-ab8a-4d7d-9384-0b01516636f4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:38.505359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4469bc-2d7b-44ee-8046-10b110870c79 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:38.524916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:38.525693 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.525977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.526251 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.527653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:38.529128 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:38.530984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 492/1012] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:38.536680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:38.541972 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:38.542812 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.543096 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.543369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] PATH_INFO: `/v3/services/compute` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.544024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:38.545853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:38.558535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:38.560003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:38.560946 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 521/1013] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:38.565853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:38.629032 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:38.655071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19b124-c078-4a02-8f0d-bb574af003cd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:38.676486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba8cde-c649-4861-b07a-2c6500699118 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:38.691558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:38.692225 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.692503 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.692773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.693613 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:38.695151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:38.697754 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:38.715742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:38.716515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.716800 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.717103 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.717803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:38.718110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:38.719487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:38.725292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 522/1014] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:38.725490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 493/1015] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:38 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:49:38.734544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:38.847478 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520ec3-a72b-4121-935c-4507e573d8c0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:38.884255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:38.884959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:38.885250 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:38.885536 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:38.886258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:38.886608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:38.888044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f13431-2db0-4d4f-ad98-4496b42d4af7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:38.901209 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 523/1016] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 14:49:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:38.901449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:39.008810 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ab98-cbb7-404d-af01-c6425165259c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:39.038370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:39.045306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e47a0-6ffe-4da1-acfa-29d3b6dfe16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:39.045993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e47a0-6ffe-4da1-acfa-29d3b6dfe16b None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:39.046589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e47a0-6ffe-4da1-acfa-29d3b6dfe16b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:39.046862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e47a0-6ffe-4da1-acfa-29d3b6dfe16b None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:39.047986 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e47a0-6ffe-4da1-acfa-29d3b6dfe16b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:39.050029 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e47a0-6ffe-4da1-acfa-29d3b6dfe16b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:39.070044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 494/1017] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:39.299300 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c95ac-41df-4cbd-99f2-fb492bf6d2fe None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:39.299857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c95ac-41df-4cbd-99f2-fb492bf6d2fe None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:39.299857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c95ac-41df-4cbd-99f2-fb492bf6d2fe None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:39.301265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 524/1018] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:39.306846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc5c2-ed4b-40da-80c4-7d63e963a2f3 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:39.307297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc5c2-ed4b-40da-80c4-7d63e963a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:39.307410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbc5c2-ed4b-40da-80c4-7d63e963a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:39.325178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-1efbc5c2-ed4b-40da-80c4-7d63e963a2f3 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:39.326408 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-1efbc5c2-ed4b-40da-80c4-7d63e963a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:39.401981 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 495/1019] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:39 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:39.424327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c95ac-41df-4cbd-99f2-fb492bf6d2fe None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:39.474435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:39.533319 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c95ac-41df-4cbd-99f2-fb492bf6d2fe None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:39.568937 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59415952-5843-474e-883a-90e8f4de55c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:39.569566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59415952-5843-474e-883a-90e8f4de55c4 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:39.569849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59415952-5843-474e-883a-90e8f4de55c4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:39.570123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59415952-5843-474e-883a-90e8f4de55c4 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:39.570844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59415952-5843-474e-883a-90e8f4de55c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:39.572232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59415952-5843-474e-883a-90e8f4de55c4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:39.596045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 525/1020] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:49:39 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:41.022158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d078d-67ca-4604-b75a-5e837d066f68 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.022692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d078d-67ca-4604-b75a-5e837d066f68 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.022692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d078d-67ca-4604-b75a-5e837d066f68 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.024168 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 496/1021] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:41.029529 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.029958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.030060 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.047443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:41.048647 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:41.121768 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 526/1022] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:41.146708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d078d-67ca-4604-b75a-5e837d066f68 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:41.261263 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d078d-67ca-4604-b75a-5e837d066f68 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:41.276040 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:41.303514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7a30-09da-4640-b110-778d28995cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:41.304332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7a30-09da-4640-b110-778d28995cb7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.304607 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7a30-09da-4640-b110-778d28995cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.304875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7a30-09da-4640-b110-778d28995cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.306114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7a30-09da-4640-b110-778d28995cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:41.307571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7a30-09da-4640-b110-778d28995cb7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:41.314156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 497/1023] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:41.315388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:41.349756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:41.406867 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9e17-d84e-4d0a-a333-669db138c281 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.407136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9e17-d84e-4d0a-a333-669db138c281 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.407429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9e17-d84e-4d0a-a333-669db138c281 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.408938 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 498/1024] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:41.413779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.414058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.414325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.427256 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ce70a-6bd2-41ba-8873-307b7466c859 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:41.431289 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:41.432551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:41.465444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:41.466133 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.466416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.466687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.467402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:41.468861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:41.491124 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 527/1025] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:49:41 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:49:41.509997 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 499/1026] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:41.534352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:41.647765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:41.672672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aef37-2f2e-40a2-9bf9-da0117f911db None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:41.713097 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b5122-765d-442d-82fd-913afa509b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:41.713943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b5122-765d-442d-82fd-913afa509b37 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.714223 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b5122-765d-442d-82fd-913afa509b37 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.714487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b5122-765d-442d-82fd-913afa509b37 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.725619 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b5122-765d-442d-82fd-913afa509b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-df2b5122-765d-442d-82fd-913afa509b37 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:41.734214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:41.735279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:41.736203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 528/1027] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:49:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:41.741561 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:41.853817 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194bf30-444c-47a6-b55c-28b54ebfaaf5 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:41.857878 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5558cbd-8b55-4bb5-b021-504570e22b66 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.858136 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5558cbd-8b55-4bb5-b021-504570e22b66 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.858438 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5558cbd-8b55-4bb5-b021-504570e22b66 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.863764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 529/1028] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:41.865145 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.865526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.867749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:41.883689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:41.884898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:41.897946 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:41.899071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:41.899546 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:41.900364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:41.901629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:41.902311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:41.904207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:41.918399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 500/1029] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:49:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:41.919717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:41.959586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 530/1030] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:41.981915 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.031893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06783bd-4292-4024-87f2-ef66b7bcda63 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.067528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.068945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.069357 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.069757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.070412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.082712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.084205 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.095197 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9abfbd-8e21-4c9b-aa91-78053dc7322f None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:42.102034 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:42.103035 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:42.104088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:42.104088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:42.104088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:42.104088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:42.104088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 501/1031] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:42.109297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.132222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.133036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.133352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.133629 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] PATH_INFO: `/v3/services/image` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.134285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.135701 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.136522 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:42.148555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:42.153555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:42.154496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 531/1032] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:49:41 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:42.156694 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.216872 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d6a7-7667-4831-8b8d-b0498db32367 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.253852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.254671 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.255043 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.255319 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.256080 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:42.256366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.257793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.258234 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28c4d4-946a-4cbc-8a8b-f7cf452f6912 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.275521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 502/1033] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:42.276565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.300039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.300712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.301002 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.301284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.301996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] RBAC: Adding query filter params (name=image) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:42.302301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.303685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.313695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 532/1034] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:42.314425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.323474 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.365552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.382787 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a418d-cc7e-4671-873a-968c7a9e9f0d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.421187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.422042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.422322 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.422610 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.424679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e690c6-3921-4d13-a498-8f812584456d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.429965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.431423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:42.446753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:42.447879 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:42.448921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 503/1035] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:42.453734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.464503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.465326 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.465612 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.465885 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.466590 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:42.466893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.468506 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.478258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 533/1036] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:42.479208 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.558844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.559537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3385d-f4e7-47c5-97f2-e597a24dddf7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.596975 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.597692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.597977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.598255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.598552 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817bcc2-787e-4d69-af8a-2ac1313b67c4 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.599095 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:42.599406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.600858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.619887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 504/1037] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:42.625491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.631877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.640661 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.641389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.641674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.641945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.642744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] RBAC: Adding query filter params (interface=public, service_id=faf4738043f14616b07b36ca25b6d648, region_id=RegionOne) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:42.643054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.644444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.650467 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 534/1038] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/endpoints?service_id=faf4738043f14616b07b36ca25b6d648&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:42.655753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:42.739535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55723bec-a359-4ee0-a015-b5ce6155bb8b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884ac65-c606-4976-8a89-b311a9bf4829 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:42.776342 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.777007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.777392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.777663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.810434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:42.811466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:42.811963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:42.812644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:42.814416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.816001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.817400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.825246 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 535/1039] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:49:42.855306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:42.858086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:42.862010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:42.978506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 505/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:42.984454 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:43.090486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279a93-584e-4890-9362-955f566500e8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:43.125746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:43.126861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.127330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.127805 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.139609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:43.141354 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:43.159412 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:43.160508 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:43.161542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:43.161542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:43.161542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:43.161542 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:43.167247 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 536/1041] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:43.169066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:43.276760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3e9e3-f555-4b46-9815-cfb60f332a3f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:43.278315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:43.313194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:43.313903 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.314189 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.314458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.315192 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:43.315496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:43.316882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:43.328735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 506/1042] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:43.334290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:43.387862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:43.439671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233396d-a9de-4262-a21a-3bc7999e4564 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:43.473521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:43.474339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.474613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.474884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.481802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:43.483249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:43.497509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:43.498515 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:43.499740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:43.501593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 537/1043] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:43.504575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:43.615018 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525f710-e9c2-42e9-b200-3345bc2a0f62 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:43.662931 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:43.663606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.663864 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.664148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.664949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:43.665223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:43.666709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcdbf6-4348-4b80-a3ab-14337a7c544f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:43.680340 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 507/1044] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:43.686323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:43.790068 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a918425-686f-4c5b-8045-5d58ef337745 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:43.796437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63d5a5-e6b4-4074-86b7-bb8374eeb3cd None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.796720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63d5a5-e6b4-4074-86b7-bb8374eeb3cd None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.797002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63d5a5-e6b4-4074-86b7-bb8374eeb3cd None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.798593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 508/1045] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:43.803569 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.803856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.804132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.806148 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:43.821907 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:43.823144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:43.830897 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:43.831752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:43.832218 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:43.832646 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:43.833665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] RBAC: Adding query filter params (role.id=ef61c989c0eb46348b817f588d6116be, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=502c9194cb0348f185dd9a5262ad1d2e) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:43.834132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:43.835759 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:43.841777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:43.848006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 538/1046] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:43 2022] GET /identity/v3/role_assignments?user.id=502c9194cb0348f185dd9a5262ad1d2e&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=ef61c989c0eb46348b817f588d6116be => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:43.915347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 509/1047] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:43.939268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:44.057657 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201deecd-1e2c-493f-8a24-4094f1cbab38 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:44.092826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:44.093680 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:44.093955 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:44.094214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:44.105094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:44.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:44.114258 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:44.115205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 539/1048] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:44.120419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:44.223217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff71c98-48c6-4615-a0f1-828f43f72699 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:44.259066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:44.259804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:44.260086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:44.260354 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:44.261111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:44.261423 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:44.262890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:44.276521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 510/1049] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:44.277890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:44.314349 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:44.383722 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537351-4c36-4d48-9990-fc4fe470609d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:44.420283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:44.421550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:44.422044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:44.422544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] PATH_INFO: `/v3/users/5436ade1b9af41dc915b3ef5b73b9d0c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:44.437350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] RBAC: Authorizing `identity:get_user(user_id=5436ade1b9af41dc915b3ef5b73b9d0c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:44.439105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:44.445729 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 540/1050] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:49:44 2022] GET /identity/v3/users/5436ade1b9af41dc915b3ef5b73b9d0c => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:44.447124 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:44.464815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:44.555256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d836a0f-8e3c-434f-8c41-3241b45c2987 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:44.590237 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:44.590944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:44.591225 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:44.591499 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:44.670023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:44.671495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:44.687271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:44.777044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 511/1051] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:44.782975 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:44.882455 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:44.893630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1b86-f304-4be9-add0-c47ca98c815e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:44.931380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73867b-189c-4385-8431-d433e32fa19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:44.932603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73867b-189c-4385-8431-d433e32fa19f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:44.933103 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73867b-189c-4385-8431-d433e32fa19f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:44.933631 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73867b-189c-4385-8431-d433e32fa19f None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:44.935236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73867b-189c-4385-8431-d433e32fa19f None admin] RBAC: Authorizing `identity:get_project(project_id=ec7e23def3094a2aa79a4423cf73b89d)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:44.936954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73867b-189c-4385-8431-d433e32fa19f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:44.944102 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 541/1052] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:49:44 2022] GET /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:44.945570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:45.050243 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0381741-d75e-4688-b7ad-85b195d7853f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:45.054334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf721fd-beba-465d-8107-e15631a22f79 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.054629 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf721fd-beba-465d-8107-e15631a22f79 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.054896 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf721fd-beba-465d-8107-e15631a22f79 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.056430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 542/1053] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:45.062154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.062455 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.062899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.065531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:45.081054 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:45.082313 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:45.089358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:45.090092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.090371 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.090641 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.091475 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=ec7e23def3094a2aa79a4423cf73b89d, user.id=5436ade1b9af41dc915b3ef5b73b9d0c) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:45.091781 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:45.093203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:45.098667 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 512/1054] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:44 2022] GET /identity/v3/role_assignments?user.id=5436ade1b9af41dc915b3ef5b73b9d0c&scope.project.id=ec7e23def3094a2aa79a4423cf73b89d&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:45.158053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 543/1055] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:45.181141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:45.294148 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14530332-1c3b-4022-8b0b-81e2ff409471 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:45.330141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:45.330953 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.331241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.331514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] PATH_INFO: `/v3/services/image` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.332174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:45.333606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:45.345574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:45.346675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:45.347631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 513/1056] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 14:49:45 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:45.353130 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:45.447564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:45.458486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a9a6d-3104-44bb-91a0-37baf1add969 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:45.493959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:45.494644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.494920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.495191 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.495940 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] RBAC: Adding query filter params (name=image) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:45.496244 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:45.497655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:45.507513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 544/1057] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:45.508816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:45.555840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:45.618453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f4a22-477f-4b0c-814e-ed9a27fd2207 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:45.661642 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:45.662326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.662603 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.662874 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.663587 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] RBAC: Adding query filter params (type=image) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:45.663888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:45.666338 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a299-b96a-4f11-be8e-f45022bf3622 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:45.672792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 514/1058] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:45.679216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:45.784289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb725b8-f296-4ebe-a09b-2bc851b64697 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:45.818675 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d39326-1614-441e-ba62-b41490b6727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:45.819344 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d39326-1614-441e-ba62-b41490b6727b None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:45.819622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d39326-1614-441e-ba62-b41490b6727b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:45.819891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d39326-1614-441e-ba62-b41490b6727b None admin] PATH_INFO: `/v3/endpoints` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:45.820635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d39326-1614-441e-ba62-b41490b6727b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:45.822030 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d39326-1614-441e-ba62-b41490b6727b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:45.845749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 545/1059] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:49:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:46.039539 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f76bd-76fb-4b9e-bbf7-511a5978408f None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:46.039749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f76bd-76fb-4b9e-bbf7-511a5978408f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:46.039992 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f76bd-76fb-4b9e-bbf7-511a5978408f None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:46.041427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 515/1060] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:46.046837 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:46.047124 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:46.047392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:46.066532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:46.067756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:46.070898 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:46.145639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 546/1061] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:46.167118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f76bd-76fb-4b9e-bbf7-511a5978408f None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:46.277934 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f76bd-76fb-4b9e-bbf7-511a5978408f None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:46.285856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:46.315525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:46.316270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:46.316535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:46.316799 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:46.327409 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:46.335739 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:46.336714 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:46.341921 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 516/1062] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:49:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:46.344786 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:46.448355 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66b39b-bd10-40dd-ac78-bcfe306f72e6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:46.483924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:46.484606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:46.484886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:46.485154 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:46.485936 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:46.486244 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:46.487637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:46.489912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:46.497559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 547/1063] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:49:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:46.503309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:46.608750 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854d84d-5afd-48c0-8b29-aab755291abe None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:46.651807 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:46.652718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:46.653003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:46.653271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:46.664794 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:46.666309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:46.684580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:46.685891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:46.686888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:46.686888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:46.686888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:46.686888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:46.687645 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 517/1064] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:46.691649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:46.797986 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e74ad8-580e-486d-a2aa-30b7f24317ce None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:46.832816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:46.833530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:46.833815 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:46.834086 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:46.834818 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:46.835127 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:46.836514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:46.848808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 548/1065] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:46.854389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:46.876093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:46.964895 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db7331-4565-4cbf-ae74-1ed6997bcb52 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.005111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.005964 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.006245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.006518 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] PATH_INFO: `/v3/users/nova` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.015280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.016746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:47.031815 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:47.032932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:47.034042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 518/1066] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:49:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:47.042043 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.107545 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:47.157763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49eef9-bca9-4693-aff8-e937ca526ede None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.194006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.194685 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.194969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.195241 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.196081 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:47.196386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.197874 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee0a80-7d5c-4bf2-ae4e-2016f6e4eb1b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.215644 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 549/1067] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:47.221272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.305104 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:47.312293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568ad05-c004-47eb-9b28-3235624d5a29 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.312583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568ad05-c004-47eb-9b28-3235624d5a29 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.312862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568ad05-c004-47eb-9b28-3235624d5a29 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.314309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 550/1068] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:47.320352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.320720 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.321100 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.331048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d5ecb-0bcf-4f41-94d5-dd339893f018 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.337713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:47.338918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:47.374767 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.375588 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.375878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.376144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.388790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.390301 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:47.408857 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:47.409888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:47.410915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:47.410915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:47.410915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:47.410915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:47.410915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 519/1069] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:47.413943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 551/1070] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:47.415639 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.436871 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.500919 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:47.527718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830422e-6913-40fc-bfab-bb9310a41123 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.557886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a69231-94f5-48a3-9371-eac31d566d2a None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.558707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:47.565525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.566204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.566485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.566846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.567620 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:47.567924 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.569422 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.584887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 520/1071] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:47.590449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.597838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.598738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.599092 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.599469 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.610305 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:47.618686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:47.619924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:47.621263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:47.625390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 552/1072] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:47.626518 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.705082 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f570e-20fe-4ef4-b8a7-4675f0b329e6 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.737743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde28f-e44e-4409-b28c-f2afd837d6eb None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.743387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.744203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.744483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.744757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.745592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:47.752180 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.754209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:47.768156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:47.769155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:47.770194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 521/1073] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:47.775980 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.777984 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.778669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.778954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.779235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.779976 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:47.780281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.781718 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.795729 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 553/1074] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:47.796996 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.836627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:47.893266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22c61b-a3a4-4a93-abd7-c1c3dab63ee4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.904966 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea20da-4798-4785-8fcf-e1d66281afc9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:47.938106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.938791 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.939074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.939351 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.940153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:47.940466 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.941964 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.945664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:47.946799 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:47.947279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:47.947736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] PATH_INFO: `/v3/users/5436ade1b9af41dc915b3ef5b73b9d0c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:47.949207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] RBAC: Authorizing `identity:get_user(user_id=5436ade1b9af41dc915b3ef5b73b9d0c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:47.950911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:47.952917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 554/1075] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/users/5436ade1b9af41dc915b3ef5b73b9d0c => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:47.957336 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 522/1076] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:47.960226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:47.960370 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:48.001447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:48.072123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99015b-7969-4980-8eb2-720e4fc26d37 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:48.074778 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328a553-a580-4490-b850-e4f4b1426d60 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:48.091181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:48.110844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df4a3e-d34c-4a70-b91b-649ad8a8452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:48.111738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4a3e-d34c-4a70-b91b-649ad8a8452a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.112021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4a3e-d34c-4a70-b91b-649ad8a8452a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.112295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4a3e-d34c-4a70-b91b-649ad8a8452a None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.113496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df4a3e-d34c-4a70-b91b-649ad8a8452a None admin] RBAC: Authorizing `identity:get_project(project_id=ec7e23def3094a2aa79a4423cf73b89d)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:48.114963 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df4a3e-d34c-4a70-b91b-649ad8a8452a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:48.116099 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cf967-db71-4580-b2b0-e526c5eac16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:48.116856 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 555/1077] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:49:47 2022] GET /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:48.117080 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cf967-db71-4580-b2b0-e526c5eac16f None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.117398 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cf967-db71-4580-b2b0-e526c5eac16f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.117671 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cf967-db71-4580-b2b0-e526c5eac16f None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/502c9194cb0348f185dd9a5262ad1d2e/roles/ef61c989c0eb46348b817f588d6116be` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.117912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70e7c4-a86b-4a34-bd56-d5b3b3f607e8 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.118200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70e7c4-a86b-4a34-bd56-d5b3b3f607e8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.118476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70e7c4-a86b-4a34-bd56-d5b3b3f607e8 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.119704 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 556/1078] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:48.122555 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70e7c4-a86b-4a34-bd56-d5b3b3f607e8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:48.123688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cf967-db71-4580-b2b0-e526c5eac16f None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=502c9194cb0348f185dd9a5262ad1d2e, role_id=ef61c989c0eb46348b817f588d6116be)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:48.125232 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cf967-db71-4580-b2b0-e526c5eac16f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:48.139077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 523/1079] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:49:47 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/502c9194cb0348f185dd9a5262ad1d2e/roles/ef61c989c0eb46348b817f588d6116be => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:49:48.140549 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.140841 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.141099 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.161259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:48.162511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:48.243672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 524/1080] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:48.248903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70e7c4-a86b-4a34-bd56-d5b3b3f607e8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:48.251864 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:48.266609 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:48.292155 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:48.293497 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.293811 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.294105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d/users/5436ade1b9af41dc915b3ef5b73b9d0c/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.297157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec7e23def3094a2aa79a4423cf73b89d, user_id=5436ade1b9af41dc915b3ef5b73b9d0c, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:48.298677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:48.315483 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 557/1081] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:49:48 2022] PUT /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d/users/5436ade1b9af41dc915b3ef5b73b9d0c/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:49:48.394930 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:48.399906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd788ede-e379-4eda-8cd4-03c61c726405 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:48.440802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:48.441504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.441792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.442026 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.522324 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:48.523675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:48.631882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:48.636438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 525/1082] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:48.642488 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:48.746040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596952c2-1ac4-48fb-8bd8-68158abe7964 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:48.781566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:48.782285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.782559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.782825 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.793199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:48.794665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e7ec97d4-53f2-43ab-a5db-08f3d7e11669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:48.812193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:48.813216 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:48.814248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:48.814248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:48.814248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:48.814248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:48.814248 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 558/1083] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:48.822288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:48.926444 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8b05a-5bd8-4a12-a848-9eb22f80f07a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:48.962151 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:48.962828 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:48.963107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:48.963381 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:48.964105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:48.964412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:48.965884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccc0c1-e3aa-4ee6-8d3a-ae9d92d98864 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:48.977591 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 526/1084] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:50.377273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5677-d01d-4765-b00b-0712625d98f6 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.377858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5677-d01d-4765-b00b-0712625d98f6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.377858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5677-d01d-4765-b00b-0712625d98f6 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.379256 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 559/1085] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:50.384802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50769-d4ce-4124-8243-3a98fd09b6e1 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.385363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50769-d4ce-4124-8243-3a98fd09b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.385363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50769-d4ce-4124-8243-3a98fd09b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.403374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-ddd50769-d4ce-4124-8243-3a98fd09b6e1 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:50.404555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-ddd50769-d4ce-4124-8243-3a98fd09b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:50.462337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:50.475564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 527/1086] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:50.499365 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d5677-d01d-4765-b00b-0712625d98f6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:50.519006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:50.522293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac62043-a09c-472c-930b-02f3064dfe31 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.522608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac62043-a09c-472c-930b-02f3064dfe31 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.522884 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac62043-a09c-472c-930b-02f3064dfe31 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.524250 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 528/1087] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:49:50.529823 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.530108 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.530383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.547087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:50.548279 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:50.612868 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d5677-d01d-4765-b00b-0712625d98f6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:50.617798 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 529/1088] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:49:50.641484 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:50.652636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:50.653471 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.653786 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.654060 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.664628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:50.673017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:50.674144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:50.675101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 560/1089] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 14:49:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:50.680336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:50.745003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdff1f-f1ae-4c01-89a6-4b905622417c None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:50.761089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:50.764753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:50.779661 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:50.780468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.780746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.781019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.785841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6707-8f4a-4ade-b5d4-416692b60abe None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:50.791303 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:49:50.799132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:50.800514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:50.801759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 530/1090] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:50.806309 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:50.823737 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:50.824471 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.824749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.825009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.825801 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:50.826116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:50.827482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:50.836057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 561/1091] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:49:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:50.841852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:50.911090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e6ca5-0e0e-472c-9046-efe32527272d None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:50.945144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c21640-ea53-439c-baea-3cc5e5cfdda9 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:50.945856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:50.946530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.946824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.947098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.947851 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:50.948157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:50.949645 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:50.958143 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 531/1092] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:49:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:50.964009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:50.980954 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:50.981811 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:50.982092 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:50.982382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:50.989523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:50.993587 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:50.995503 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.007905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:51.012589 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:51.013781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:51.015012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:51.015012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:51.015012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:51.015012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:51.015012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 562/1093] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:51.019636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.066963 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb1fe-1656-4f20-9b18-05a6d4e8e939 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.100183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.101164 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.101474 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.101742 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] PATH_INFO: `/v3/users/5436ade1b9af41dc915b3ef5b73b9d0c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.102971 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] RBAC: Authorizing `identity:get_user(user_id=5436ade1b9af41dc915b3ef5b73b9d0c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.104412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.106133 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 532/1094] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:49:50 2022] GET /identity/v3/users/5436ade1b9af41dc915b3ef5b73b9d0c => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:51.112009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.117687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd4750-c5e4-4103-b7f5-645f6995a17d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.151504 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.152230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.152511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.152790 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.153557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:51.153855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.155262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.167402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 563/1095] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:51.173115 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.209550 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63efcb2-8ce5-4036-a467-c82e0a1a6e28 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.219661 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.224018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.243713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.244363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.244639 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.244891 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.273430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651d687-0f99-414a-be25-7f538767f775 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.313628 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.313916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.314442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.314698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.314995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.315376 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.323155 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.324601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:49:51.339181 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:51.340521 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:49:51.342056 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 564/1096] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:51.346237 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.420790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 533/1097] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:51.426566 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.438056 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.445945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fba3a-d04e-4ac2-810e-0c4f3f79a9ac None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.457263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.481026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.481742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.482089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.482365 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.483191 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:51.483527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.484990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.502400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 565/1098] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:51.508339 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.528932 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee20d-7780-4da6-bf6a-4892c194ff3a None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.561743 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.562669 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.562951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.563219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.564400 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] RBAC: Authorizing `identity:get_project(project_id=ec7e23def3094a2aa79a4423cf73b89d)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.565913 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.567627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 534/1099] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:51.573640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.606407 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68015-d382-42eb-ab36-0e471089de7b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.643137 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.643799 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.644085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.644358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.655560 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.678438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3eaf2-800f-4c63-b88b-15861431d030 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.679798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:51.712691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.713570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.715635 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.715748 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.715845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.715845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.715845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=ec7e23def3094a2aa79a4423cf73b89d, user.id=5436ade1b9af41dc915b3ef5b73b9d0c) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:51.715845 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.717257 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.723203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 535/1100] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/role_assignments?user.id=5436ade1b9af41dc915b3ef5b73b9d0c&scope.project.id=ec7e23def3094a2aa79a4423cf73b89d&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:49:51.829862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 566/1101] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 14:49:51.833795 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:51.941310 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7d75b-6841-4bb4-b1d3-fab1af9ee9fb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:51.974271 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:51.975074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:51.975348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:51.975614 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:51.986104 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:51.987536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:51.989330 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:49:52.005803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:52.007009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:52.008072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:49:52.008072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:49:52.008072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:49:52.008072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:52.008072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 536/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:49:52.012742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:52.081120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:52.116397 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec428a-cffe-439f-abed-7a1763d11600 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:52.153841 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:52.154516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:52.154796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:52.155066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:52.155792 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:52.156096 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:52.157516 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:52.169392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 567/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:52.175172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:52.282631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db266bc-1e2a-4acf-9828-728a07df395e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:52.317878 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:52.318694 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:52.318978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:52.319247 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:52.326411 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:52.327872 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:49:52.342458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:49:52.343551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:49:52.344542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 537/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:49:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:49:52.349679 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:52.457049 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051368e4-633e-43df-a65f-f88c3b654259 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:52.484819 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:52.494080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:52.494756 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:52.495036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:52.495307 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:52.496131 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:52.496439 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:52.497913 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf00fc6-53e5-430d-a56a-40caa774b832 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:52.510749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 568/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:49:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:52.517039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:52.576476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:52.625836 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0a928-f07e-4f05-b023-b6624c6ca241 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:52.663579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:52.664245 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:52.664528 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:52.664800 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:52.665684 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] RBAC: Adding query filter params (role.id=ef61c989c0eb46348b817f588d6116be, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=502c9194cb0348f185dd9a5262ad1d2e) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:52.665990 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:52.667523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedb08f-7538-41c6-9f99-a93b817b8380 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:52.673638 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 538/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:52 2022] GET /identity/v3/role_assignments?user.id=502c9194cb0348f185dd9a5262ad1d2e&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=ef61c989c0eb46348b817f588d6116be => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:53.991925 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c591-5b0e-4280-b65e-d4ab9cec3f84 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:53.992777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c591-5b0e-4280-b65e-d4ab9cec3f84 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:53.992777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c591-5b0e-4280-b65e-d4ab9cec3f84 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:53.994959 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 569/1107] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:54.004352 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:54.004806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:54.004903 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:54.027090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:54.028283 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:54.099424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 539/1108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:54.124809 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c591-5b0e-4280-b65e-d4ab9cec3f84 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:54.247101 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c591-5b0e-4280-b65e-d4ab9cec3f84 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:54.282121 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f68fa-6e8b-4612-b186-97ce2ef4ced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:54.282907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f68fa-6e8b-4612-b186-97ce2ef4ced2 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:54.283178 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f68fa-6e8b-4612-b186-97ce2ef4ced2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:54.283444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f68fa-6e8b-4612-b186-97ce2ef4ced2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:54.284750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f68fa-6e8b-4612-b186-97ce2ef4ced2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:54.286214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f68fa-6e8b-4612-b186-97ce2ef4ced2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:54.287827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 570/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:54.293753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:54.346068 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:54.412169 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36370347-b556-4195-80ff-c33d9be482a0 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:54.447736 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571c57c-4e5f-442e-9afc-6aa9a208204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:54.448403 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571c57c-4e5f-442e-9afc-6aa9a208204c None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:54.448680 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571c57c-4e5f-442e-9afc-6aa9a208204c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:54.449039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571c57c-4e5f-442e-9afc-6aa9a208204c None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:54.449793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571c57c-4e5f-442e-9afc-6aa9a208204c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:54.451235 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571c57c-4e5f-442e-9afc-6aa9a208204c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:54.473966 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 540/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:49:54 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:49:56.705112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaa747-6fff-46e9-b02f-2e0ed270199a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:56.705737 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaa747-6fff-46e9-b02f-2e0ed270199a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:56.705737 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaa747-6fff-46e9-b02f-2e0ed270199a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:56.707112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 571/1111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:49:56.713905 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:56.714372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:56.714481 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:56.744068 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:49:56.745996 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:49:56.825279 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 541/1112] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:49:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:49:56.849839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaa747-6fff-46e9-b02f-2e0ed270199a None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:56.918177 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:56.965232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaa747-6fff-46e9-b02f-2e0ed270199a None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:57.004891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:57.005792 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:57.006073 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:57.006345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] PATH_INFO: `/v3/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:57.007842 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] RBAC: Authorizing `identity:get_role(role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:57.009231 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:57.010671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 572/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:49:56 2022] GET /identity/v3/roles/75add8e0c0454b3a8681eab45a26244c => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:57.016907 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:57.060202 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:57.136967 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abdfea-583a-47ae-ad14-261d0128e2df None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:57.174621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:57.175448 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:57.175728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:57.176000 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] PATH_INFO: `/v3/users/44092360ea554c4f97afc875f3e3e1f6` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:57.189750 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] RBAC: Authorizing `identity:get_user(user_id=44092360ea554c4f97afc875f3e3e1f6)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:57.191306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:57.192993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 542/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:49:57 2022] GET /identity/v3/users/44092360ea554c4f97afc875f3e3e1f6 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:57.199928 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:57.311669 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcc442-3e3a-4333-b128-3513cf7299ec None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:57.334758 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:57.347535 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:57.348185 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:57.348462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:57.348737 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:57.423303 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:57.424833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:57.536444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 573/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:49:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:49:57.542816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:57.637810 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:57.648455 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138092c-4ed7-4f4c-a120-40c22aa06711 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:57.684445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617b2b7-d7ff-4008-bc37-faa95e743abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:57.685285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617b2b7-d7ff-4008-bc37-faa95e743abd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:57.685573 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617b2b7-d7ff-4008-bc37-faa95e743abd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:57.685849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617b2b7-d7ff-4008-bc37-faa95e743abd None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:57.687115 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617b2b7-d7ff-4008-bc37-faa95e743abd None admin] RBAC: Authorizing `identity:get_project(project_id=ec7e23def3094a2aa79a4423cf73b89d)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:57.688557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617b2b7-d7ff-4008-bc37-faa95e743abd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:57.690314 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 543/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:49:57 2022] GET /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:49:57.696648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:49:57.722094 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:49:57.809635 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b64061-de94-4b1c-bf13-eae24ad130bb None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:49:57.849380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:49:57.850060 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:49:57.850341 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:49:57.850617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:49:57.851458 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] RBAC: Adding query filter params (role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=ec7e23def3094a2aa79a4423cf73b89d, user.id=44092360ea554c4f97afc875f3e3e1f6) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:49:57.851769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:49:57.853220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d2fb5-bdfa-497a-9d19-664bd7f86176 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:49:57.859153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 574/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:49:57 2022] GET /identity/v3/role_assignments?user.id=44092360ea554c4f97afc875f3e3e1f6&scope.project.id=ec7e23def3094a2aa79a4423cf73b89d&role.id=75add8e0c0454b3a8681eab45a26244c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:00.032801 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbb2c8-753c-4051-b8af-af26404f41f6 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:00.033720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbb2c8-753c-4051-b8af-af26404f41f6 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:00.033720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbb2c8-753c-4051-b8af-af26404f41f6 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:00.035774 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 544/1118] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:00.044571 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:00.045032 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:00.045140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:00.068199 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:00.069514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:00.138915 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 575/1119] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:00.162446 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbb2c8-753c-4051-b8af-af26404f41f6 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:00.275483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbb2c8-753c-4051-b8af-af26404f41f6 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:00.314537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:00.315332 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:00.315605 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:00.315876 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] PATH_INFO: `/v3/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:00.317449 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] RBAC: Authorizing `identity:get_role(role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:00.318842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:00.320203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 545/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:00 2022] GET /identity/v3/roles/75add8e0c0454b3a8681eab45a26244c => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:00.328662 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:00.353614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:00.440033 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df629936-2ff0-44ab-8f2e-5ca3d1912eb1 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:00.478163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:00.479527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:00.480017 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:00.480353 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] PATH_INFO: `/v3/users/44092360ea554c4f97afc875f3e3e1f6` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:00.481665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] RBAC: Authorizing `identity:get_user(user_id=44092360ea554c4f97afc875f3e3e1f6)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:00.483119 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:00.484750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 576/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:00 2022] GET /identity/v3/users/44092360ea554c4f97afc875f3e3e1f6 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:00.490618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:00.523330 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:00.598864 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f024e3e-ca90-4c51-9875-93d8f00b2808 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:00.644977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e114d4-123b-42fd-8ff2-e630ef28553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:00.646282 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e114d4-123b-42fd-8ff2-e630ef28553f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:00.646558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e114d4-123b-42fd-8ff2-e630ef28553f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:00.646827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e114d4-123b-42fd-8ff2-e630ef28553f None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:00.648086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e114d4-123b-42fd-8ff2-e630ef28553f None admin] RBAC: Authorizing `identity:get_project(project_id=ec7e23def3094a2aa79a4423cf73b89d)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:00.649582 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e114d4-123b-42fd-8ff2-e630ef28553f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:00.651300 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 546/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:00 2022] GET /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:00.657434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:00.745588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:00.763862 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67f3d-3f91-4ab8-810f-caea41993d11 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:00.799329 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dc302-984f-4a8f-b340-1304cb67bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:00.800219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dc302-984f-4a8f-b340-1304cb67bfab None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:00.800496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dc302-984f-4a8f-b340-1304cb67bfab None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:00.800770 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dc302-984f-4a8f-b340-1304cb67bfab None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d/users/44092360ea554c4f97afc875f3e3e1f6/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:00.803161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dc302-984f-4a8f-b340-1304cb67bfab None admin] RBAC: Authorizing `identity:create_grant(project_id=ec7e23def3094a2aa79a4423cf73b89d, user_id=44092360ea554c4f97afc875f3e3e1f6, role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:00.804594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dc302-984f-4a8f-b340-1304cb67bfab None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:00.823856 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 577/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:50:00 2022] PUT /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d/users/44092360ea554c4f97afc875f3e3e1f6/roles/75add8e0c0454b3a8681eab45a26244c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:50:03.014193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f34be-7c58-4af0-928f-f13ec6374aaf None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:03.015057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f34be-7c58-4af0-928f-f13ec6374aaf None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:03.015057 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f34be-7c58-4af0-928f-f13ec6374aaf None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:03.017219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 547/1124] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:50:03.023278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:03.023728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:03.023826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:03.041368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:03.042602 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:03.111144 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 578/1125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:03.135454 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f34be-7c58-4af0-928f-f13ec6374aaf None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:03.242896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:03.255530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f34be-7c58-4af0-928f-f13ec6374aaf None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:03.302936 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:03.304248 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:03.304689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:03.305147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] PATH_INFO: `/v3/roles/75add8e0c0454b3a8681eab45a26244c` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:03.307871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] RBAC: Authorizing `identity:get_role(role_id=75add8e0c0454b3a8681eab45a26244c)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:03.309513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:03.310896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 548/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:03 2022] GET /identity/v3/roles/75add8e0c0454b3a8681eab45a26244c => generated 223 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:03.316957 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:03.398400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:03.425330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8389b9-b276-4b14-9477-cf521886a8f9 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:03.461853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:03.462726 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:03.463006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:03.463278 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] PATH_INFO: `/v3/users/44092360ea554c4f97afc875f3e3e1f6` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:03.464816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] RBAC: Authorizing `identity:get_user(user_id=44092360ea554c4f97afc875f3e3e1f6)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:03.466292 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:03.467924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 579/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:03 2022] GET /identity/v3/users/44092360ea554c4f97afc875f3e3e1f6 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:03.474511 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:03.584317 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49be0b-f73a-4726-bb24-42bc4680b122 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:03.620074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:03.620759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:03.621044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:03.621352 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:03.653545 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:03.694989 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:03.696458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:03.798130 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 549/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:50:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:50:03.807663 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:03.922714 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f2357-5305-49ab-9d6f-61963b9cd10b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:03.959434 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428bc4f-cc78-433d-b8c1-94bf3580f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:03.960299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428bc4f-cc78-433d-b8c1-94bf3580f9b5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:03.960584 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428bc4f-cc78-433d-b8c1-94bf3580f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:03.960858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428bc4f-cc78-433d-b8c1-94bf3580f9b5 None admin] PATH_INFO: `/v3/projects/ec7e23def3094a2aa79a4423cf73b89d` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:03.962082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428bc4f-cc78-433d-b8c1-94bf3580f9b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ec7e23def3094a2aa79a4423cf73b89d)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:03.963560 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428bc4f-cc78-433d-b8c1-94bf3580f9b5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:03.965430 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 580/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:03 2022] GET /identity/v3/projects/ec7e23def3094a2aa79a4423cf73b89d => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:03.975553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:04.051313 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:04.090636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf324-9c53-4fb1-9192-44733bae1ec5 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:04.125950 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:04.126603 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:04.126883 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:04.127157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:04.127991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] RBAC: Adding query filter params (role.id=75add8e0c0454b3a8681eab45a26244c, scope.project.id=ec7e23def3094a2aa79a4423cf73b89d, user.id=44092360ea554c4f97afc875f3e3e1f6) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:04.128302 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:04.129782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8786270-4370-4e37-8923-6aa402cb581b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:04.135843 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 550/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:50:03 2022] GET /identity/v3/role_assignments?user.id=44092360ea554c4f97afc875f3e3e1f6&scope.project.id=ec7e23def3094a2aa79a4423cf73b89d&role.id=75add8e0c0454b3a8681eab45a26244c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:06.376265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dccb9-6aa7-488f-b186-d5f00d0a9847 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:06.377193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dccb9-6aa7-488f-b186-d5f00d0a9847 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:06.377193 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dccb9-6aa7-488f-b186-d5f00d0a9847 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:06.379382 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 581/1131] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:06.385920 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:06.386579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:06.386689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:06.405952 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:06.407204 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:06.484127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 551/1132] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:06.507746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dccb9-6aa7-488f-b186-d5f00d0a9847 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:06.525399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:06.625031 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dccb9-6aa7-488f-b186-d5f00d0a9847 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:06.673230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfebe-bfeb-4803-b158-dd666add88e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:06.674067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfebe-bfeb-4803-b158-dd666add88e1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:06.674348 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfebe-bfeb-4803-b158-dd666add88e1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:06.674618 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfebe-bfeb-4803-b158-dd666add88e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:06.675965 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfebe-bfeb-4803-b158-dd666add88e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:06.677482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfebe-bfeb-4803-b158-dd666add88e1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:06.679134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 582/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:50:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:06.685277 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:06.746505 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:06.790366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cc458-a7ac-4444-8186-2663b0a38cc0 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:06.828477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db2da6-8598-4a81-a05f-478f2afb7898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:06.829139 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db2da6-8598-4a81-a05f-478f2afb7898 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:06.829456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db2da6-8598-4a81-a05f-478f2afb7898 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:06.829726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db2da6-8598-4a81-a05f-478f2afb7898 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:06.830445 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db2da6-8598-4a81-a05f-478f2afb7898 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:06.831909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db2da6-8598-4a81-a05f-478f2afb7898 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:06.859404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 552/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:50:06 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:09.116110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b9796-35b1-4749-a27f-65eecf495e33 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:09.116671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b9796-35b1-4749-a27f-65eecf495e33 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:09.116671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b9796-35b1-4749-a27f-65eecf495e33 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:09.118104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 583/1135] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:09.124640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:09.125466 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:09.125653 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:09.153853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:09.155060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:09.227418 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 553/1136] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:09.259337 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b9796-35b1-4749-a27f-65eecf495e33 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:09.353930 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:09.375723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b9796-35b1-4749-a27f-65eecf495e33 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:09.416950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6767d-e346-458e-8e7b-ab251203922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:09.417808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6767d-e346-458e-8e7b-ab251203922a None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:09.418109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6767d-e346-458e-8e7b-ab251203922a None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:09.418369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6767d-e346-458e-8e7b-ab251203922a None admin] PATH_INFO: `/v3/domains/default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:09.419695 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6767d-e346-458e-8e7b-ab251203922a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:09.421153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6767d-e346-458e-8e7b-ab251203922a None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:09.422831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 584/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:50:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:09.431658 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:09.476619 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:09.541264 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96774-33fa-45cd-a716-80b9d757331f None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:09.576491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3041e5-c3d4-4bca-a25e-c58893c7f391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:09.577161 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3041e5-c3d4-4bca-a25e-c58893c7f391 None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:09.577477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3041e5-c3d4-4bca-a25e-c58893c7f391 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:09.577753 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3041e5-c3d4-4bca-a25e-c58893c7f391 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:09.578468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3041e5-c3d4-4bca-a25e-c58893c7f391 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:09.579927 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3041e5-c3d4-4bca-a25e-c58893c7f391 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:09.605800 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 554/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:50:09 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:11.878712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaeebf-5b08-4158-ab8a-81b348d2a69e None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:11.879611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaeebf-5b08-4158-ab8a-81b348d2a69e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:11.879611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaeebf-5b08-4158-ab8a-81b348d2a69e None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:11.881914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 585/1139] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:11.889102 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:11.889608 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:11.889725 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:11.908257 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:11.909419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:11.980988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 555/1140] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:12.018797 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaeebf-5b08-4158-ab8a-81b348d2a69e None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:12.150755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaeebf-5b08-4158-ab8a-81b348d2a69e None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:12.185925 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:12.186659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:12.186939 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:12.187207 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:12.198050 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:12.205781 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:12.206960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:50:12.208242 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 586/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:50:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:12.215048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:12.244072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:12.334145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0166d8e-032b-4c6e-9ed3-4c9be16e21cb None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:12.376290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:12.376970 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:12.377287 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:12.377552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:12.378306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:12.378615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:12.380008 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:12.388695 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 556/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:50:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:12.394883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:12.408352 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:12.503772 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37951a4-3815-453c-a543-94b1c63f7926 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:12.539637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69285386-2708-4227-a7de-28e216ac474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:12.540513 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69285386-2708-4227-a7de-28e216ac474e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:12.540764 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69285386-2708-4227-a7de-28e216ac474e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:12.541047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69285386-2708-4227-a7de-28e216ac474e None admin] PATH_INFO: `/v3/users/428dc3a692264816b0df57596d4f0c7d` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:12.554406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69285386-2708-4227-a7de-28e216ac474e None admin] RBAC: Authorizing `identity:get_user(user_id=428dc3a692264816b0df57596d4f0c7d)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:12.555863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69285386-2708-4227-a7de-28e216ac474e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:12.557543 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 587/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:12 2022] GET /identity/v3/users/428dc3a692264816b0df57596d4f0c7d => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:12.564040 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:12.672773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:12.687871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933173a4-e221-4d3a-96b9-e0fce7e9a79f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:12.729542 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:12.730297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:12.730577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:12.730847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:12.808803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:12.810231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:12.896497 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:12.914050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 557/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:50:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:50:12.923494 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69285386-2708-4227-a7de-28e216ac474e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:12.999486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:13.035281 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69285386-2708-4227-a7de-28e216ac474e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:13.072711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600e2e2-41b0-4cd9-8592-1d09a32f7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:13.073715 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600e2e2-41b0-4cd9-8592-1d09a32f7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:13.074063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600e2e2-41b0-4cd9-8592-1d09a32f7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:13.074466 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600e2e2-41b0-4cd9-8592-1d09a32f7fb0 None admin] PATH_INFO: `/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:13.076036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600e2e2-41b0-4cd9-8592-1d09a32f7fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d5c850b9b44f8c8bb73f1da617039f)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:13.078036 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600e2e2-41b0-4cd9-8592-1d09a32f7fb0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:13.080467 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 588/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:12 2022] GET /identity/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:13.092660 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:13.210697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e499e-f2fa-4930-9a5d-046cad54a010 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:13.244334 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:13.244995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:13.245323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:13.245597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:13.246432 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=f3d5c850b9b44f8c8bb73f1da617039f, user.id=428dc3a692264816b0df57596d4f0c7d) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:13.246744 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:13.248194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d177ad-48b7-46d3-ba51-a2dc1c219bf2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:13.253502 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 558/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:50:13 2022] GET /identity/v3/role_assignments?user.id=428dc3a692264816b0df57596d4f0c7d&scope.project.id=f3d5c850b9b44f8c8bb73f1da617039f&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:15.417804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf833df-2327-440f-b08f-fecbcf291016 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:15.418418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf833df-2327-440f-b08f-fecbcf291016 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:15.418418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf833df-2327-440f-b08f-fecbcf291016 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:15.419735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 589/1147] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:15.426174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:15.426979 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:15.427136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:15.453273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:15.454525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:15.503842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:15.529551 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 559/1148] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:15.552230 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf833df-2327-440f-b08f-fecbcf291016 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:15.658593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf833df-2327-440f-b08f-fecbcf291016 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:15.693396 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:15.694114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:15.694387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:15.694648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:15.696626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:15.706296 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:15.713971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:15.715163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:50:15.716269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 590/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:50:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:15.722107 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:15.829260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699957e-1b02-45ce-bb55-b7af93aff0f7 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:15.864859 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:15.865587 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:15.865861 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:15.866126 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:15.866873 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:15.867175 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:15.868597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:15.877157 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 560/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:50:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:15.883211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:15.993457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927dbe0-2340-4c73-85e4-5dc9294a170e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:16.029775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:16.030649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:16.030929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:16.031202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] PATH_INFO: `/v3/users/428dc3a692264816b0df57596d4f0c7d` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:16.032517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] RBAC: Authorizing `identity:get_user(user_id=428dc3a692264816b0df57596d4f0c7d)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:16.033997 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:16.035636 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 591/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:15 2022] GET /identity/v3/users/428dc3a692264816b0df57596d4f0c7d => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:16.041770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:16.093338 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:16.161957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661365a-9d2c-4c66-be02-59d4aa93cb7b None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:16.197141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6a17-45d0-4220-925c-0fa53318cead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:16.198048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6a17-45d0-4220-925c-0fa53318cead None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:16.198383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6a17-45d0-4220-925c-0fa53318cead None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:16.198662 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6a17-45d0-4220-925c-0fa53318cead None admin] PATH_INFO: `/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:16.199869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6a17-45d0-4220-925c-0fa53318cead None admin] RBAC: Authorizing `identity:get_project(project_id=f3d5c850b9b44f8c8bb73f1da617039f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:16.202144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6a17-45d0-4220-925c-0fa53318cead None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:16.205039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 561/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:16 2022] GET /identity/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:16.214667 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:16.289057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:16.335191 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903f4b-c744-4cf3-a4a8-a81c37e74bee None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:16.372475 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67184f8e-758b-4bc6-a243-db2f088457bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:16.373399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67184f8e-758b-4bc6-a243-db2f088457bc None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:16.373690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67184f8e-758b-4bc6-a243-db2f088457bc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:16.373967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67184f8e-758b-4bc6-a243-db2f088457bc None admin] PATH_INFO: `/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f/users/428dc3a692264816b0df57596d4f0c7d/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:16.376066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67184f8e-758b-4bc6-a243-db2f088457bc None admin] RBAC: Authorizing `identity:create_grant(project_id=f3d5c850b9b44f8c8bb73f1da617039f, user_id=428dc3a692264816b0df57596d4f0c7d, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:16.377547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67184f8e-758b-4bc6-a243-db2f088457bc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:16.393460 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 592/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:50:16 2022] PUT /identity/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f/users/428dc3a692264816b0df57596d4f0c7d/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:50:18.597324 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e779b7-8b39-4cc5-bf14-1be05f9d18ea None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:18.598267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e779b7-8b39-4cc5-bf14-1be05f9d18ea None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:18.598267 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e779b7-8b39-4cc5-bf14-1be05f9d18ea None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:18.600425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 562/1154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:18.606129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:18.606576 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:18.606711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:18.624369 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:18.625609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:18.699377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 593/1155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:18.723863 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e779b7-8b39-4cc5-bf14-1be05f9d18ea None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:18.836914 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e779b7-8b39-4cc5-bf14-1be05f9d18ea None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:18.850090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:18.872793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:18.873607 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:18.873882 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:18.874144 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:18.884770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:18.892477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:18.893702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:50:18.894797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 563/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:18.900287 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:18.956320 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:19.006709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243f9d0-1985-4600-97df-136d8c0520c6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:19.042865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:19.043530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:19.043806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:19.044204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:19.044956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:19.045282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:19.046659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:19.054955 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 594/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:50:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:19.060837 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:19.166286 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3c14-40f2-4687-8a26-832932e126ae None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:19.201042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:19.201947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:19.202219 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:19.202487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] PATH_INFO: `/v3/users/428dc3a692264816b0df57596d4f0c7d` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:19.203806 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] RBAC: Authorizing `identity:get_user(user_id=428dc3a692264816b0df57596d4f0c7d)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:19.205278 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:19.206909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 564/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:19 2022] GET /identity/v3/users/428dc3a692264816b0df57596d4f0c7d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:19.213642 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:19.331070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703ed4f-ef94-4725-b87e-213507aa5e62 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:19.344911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:19.367760 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:19.368477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:19.368761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:19.369034 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:19.441985 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:19.444136 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:19.556973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 595/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:50:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:50:19.563256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:19.608555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:19.676904 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c5014-e83f-4e06-b6a6-865f6599ccdb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:19.713112 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a738-2884-41fb-a5d5-930d88a5bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:19.714021 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a738-2884-41fb-a5d5-930d88a5bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:19.714299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a738-2884-41fb-a5d5-930d88a5bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:19.714581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a738-2884-41fb-a5d5-930d88a5bdf6 None admin] PATH_INFO: `/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:19.715929 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a738-2884-41fb-a5d5-930d88a5bdf6 None admin] RBAC: Authorizing `identity:get_project(project_id=f3d5c850b9b44f8c8bb73f1da617039f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:19.717421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a738-2884-41fb-a5d5-930d88a5bdf6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:19.719145 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 565/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:19 2022] GET /identity/v3/projects/f3d5c850b9b44f8c8bb73f1da617039f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:19.726384 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:19.749425 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:19.836948 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3cadf-e040-4014-8263-90c737ca05cf None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:19.876085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:19.876731 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:19.877020 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:19.877330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:19.878149 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=f3d5c850b9b44f8c8bb73f1da617039f, user.id=428dc3a692264816b0df57596d4f0c7d) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:19.878463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:19.879909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c4262-87ca-4580-8a65-63eb4c6b6004 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:19.886067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 596/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:50:19 2022] GET /identity/v3/role_assignments?user.id=428dc3a692264816b0df57596d4f0c7d&scope.project.id=f3d5c850b9b44f8c8bb73f1da617039f&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:22.120642 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ec2a-8bc4-4f6c-8f30-bc238fc879ab None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:22.121803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ec2a-8bc4-4f6c-8f30-bc238fc879ab None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:22.121803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ec2a-8bc4-4f6c-8f30-bc238fc879ab None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:22.124121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 566/1162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:22.132511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:22.132997 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:22.132997 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:22.150990 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:22.152227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:22.221944 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 597/1163] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:50:22.248011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ec2a-8bc4-4f6c-8f30-bc238fc879ab None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:22.367948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ec2a-8bc4-4f6c-8f30-bc238fc879ab None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:22.377429 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:22.405469 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:22.406123 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:22.406406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:22.406681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:22.478058 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:22.480477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:22.583276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 567/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:50:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:50:22.589903 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:22.600852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:22.705283 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0e5d3-d1f1-4193-9f8e-0068225b9a74 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:22.750018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:22.750833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:22.751110 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:22.751375 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:22.759170 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:22.760622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-f246cabd-ff06-4947-8fea-2da6c2bc7eed None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:22.776070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:22.777549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:50:22.778765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 598/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:50:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:50:22.785156 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:22.796890 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:22.892746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764c7c-3794-487c-b9ab-fefd3d953915 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:22.938853 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:22.939514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:22.939792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:22.940060 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:22.940788 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:22.941091 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:22.942497 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03507fb4-2945-4a56-a6ad-9c3477ded07c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:22.949988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 568/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 14:50:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:25.140704 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c153d-1e88-4cd1-ab5f-590e84a070a8 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:25.141659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c153d-1e88-4cd1-ab5f-590e84a070a8 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:25.141659 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c153d-1e88-4cd1-ab5f-590e84a070a8 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:25.143877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 599/1167] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:25.149866 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506755fd-3cf7-48a8-bfec-31e1d51f30ca None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:25.150326 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506755fd-3cf7-48a8-bfec-31e1d51f30ca None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:25.150433 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506755fd-3cf7-48a8-bfec-31e1d51f30ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:25.169492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-506755fd-3cf7-48a8-bfec-31e1d51f30ca None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:25.170705 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-506755fd-3cf7-48a8-bfec-31e1d51f30ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:25.240269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 569/1168] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:25.263918 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c153d-1e88-4cd1-ab5f-590e84a070a8 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:25.349784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:25.386289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c153d-1e88-4cd1-ab5f-590e84a070a8 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:25.428532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a965afb-e953-4c2d-bb88-bb016abef9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:25.429149 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a965afb-e953-4c2d-bb88-bb016abef9fe None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:25.429471 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a965afb-e953-4c2d-bb88-bb016abef9fe None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:25.429744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a965afb-e953-4c2d-bb88-bb016abef9fe None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:25.430495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a965afb-e953-4c2d-bb88-bb016abef9fe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:25.431967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a965afb-e953-4c2d-bb88-bb016abef9fe None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:25.452139 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 600/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Thu May 19 14:50:25 2022] POST /identity/v3/domains => generated 267 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:27.679586 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb12f2f-497e-4c0b-ac2e-a4faee3fb57c None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:27.680490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb12f2f-497e-4c0b-ac2e-a4faee3fb57c None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:27.680676 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb12f2f-497e-4c0b-ac2e-a4faee3fb57c None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:27.682906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 570/1170] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:27.691950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:27.692423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:27.692541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:27.712550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:27.713763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:27.783814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 601/1171] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:27.811459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb12f2f-497e-4c0b-ac2e-a4faee3fb57c None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:27.814622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:27.920564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb12f2f-497e-4c0b-ac2e-a4faee3fb57c None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:27.957108 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257da4a-2014-4e9f-a1e8-7157e910a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:27.958006 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257da4a-2014-4e9f-a1e8-7157e910a217 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:27.958280 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257da4a-2014-4e9f-a1e8-7157e910a217 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:27.958548 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257da4a-2014-4e9f-a1e8-7157e910a217 None admin] PATH_INFO: `/v3/domains/4c4a6a8574374561bc388054b4f2acd3` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:27.969937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257da4a-2014-4e9f-a1e8-7157e910a217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4c4a6a8574374561bc388054b4f2acd3)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:27.971393 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257da4a-2014-4e9f-a1e8-7157e910a217 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:27.973031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 571/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 14:50:27 2022] GET /identity/v3/domains/4c4a6a8574374561bc388054b4f2acd3 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:27.979095 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:28.047433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:28.101484 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc634d-4c96-4934-9d63-2e8b82b18d15 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:28.146739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a07b-6bad-4b7b-af2c-69e2451835b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:28.147401 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a07b-6bad-4b7b-af2c-69e2451835b2 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:28.147677 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a07b-6bad-4b7b-af2c-69e2451835b2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:28.147947 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a07b-6bad-4b7b-af2c-69e2451835b2 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:28.148651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a07b-6bad-4b7b-af2c-69e2451835b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:28.150116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a07b-6bad-4b7b-af2c-69e2451835b2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:28.178587 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 602/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:50:27 2022] POST /identity/v3/projects => generated 361 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:30.450690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b0fc1-5450-441c-904a-7f3bf260c92a None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:30.451631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b0fc1-5450-441c-904a-7f3bf260c92a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:30.451631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b0fc1-5450-441c-904a-7f3bf260c92a None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:30.453844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 572/1174] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:30.460399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:30.460858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:30.460965 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:30.485520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:30.486735 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:30.558026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 603/1175] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:30.588862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b0fc1-5450-441c-904a-7f3bf260c92a None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:30.659822 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:30.700940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b0fc1-5450-441c-904a-7f3bf260c92a None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:30.735937 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e191-d36f-4475-9860-0ecc9d3527d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:30.736802 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e191-d36f-4475-9860-0ecc9d3527d2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:30.737073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e191-d36f-4475-9860-0ecc9d3527d2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:30.737365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e191-d36f-4475-9860-0ecc9d3527d2 None admin] PATH_INFO: `/v3/domains/4c4a6a8574374561bc388054b4f2acd3` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:30.738688 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e191-d36f-4475-9860-0ecc9d3527d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4c4a6a8574374561bc388054b4f2acd3)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:30.740119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e191-d36f-4475-9860-0ecc9d3527d2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:30.741773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 573/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 14:50:30 2022] GET /identity/v3/domains/4c4a6a8574374561bc388054b4f2acd3 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:30.747920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:30.797905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:30.878173 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2cb80-678e-4816-9f28-46e9e158bffe None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:30.914895 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915035e-14ff-45aa-b8f1-44f25ccdc9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:30.915562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915035e-14ff-45aa-b8f1-44f25ccdc9b2 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:30.915843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915035e-14ff-45aa-b8f1-44f25ccdc9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:30.916117 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915035e-14ff-45aa-b8f1-44f25ccdc9b2 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:30.916838 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915035e-14ff-45aa-b8f1-44f25ccdc9b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:30.918359 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915035e-14ff-45aa-b8f1-44f25ccdc9b2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:30.941995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 604/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:50:30 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:50:33.181989 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f072be-86cc-4f1b-9a9a-0371e8118aaa None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:33.182883 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f072be-86cc-4f1b-9a9a-0371e8118aaa None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:33.182883 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f072be-86cc-4f1b-9a9a-0371e8118aaa None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:33.184465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 574/1178] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:33.190995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:33.191438 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:33.191570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:33.218332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:33.219544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:33.290833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 605/1179] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:33.320740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f072be-86cc-4f1b-9a9a-0371e8118aaa None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:33.427943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f072be-86cc-4f1b-9a9a-0371e8118aaa None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:33.461495 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:33.468231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:33.469100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:33.469413 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:33.469689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:33.480752 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:33.488459 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:33.489712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:33.490835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:50:33.493592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 575/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:50:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:33.495808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:33.525201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:33.600890 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b3aa7-93d8-4440-adc4-c3287d39633c None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:33.641378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:33.641523 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:33.641763 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:33.642029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:33.642784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:33.643083 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:33.644458 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:33.654650 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 606/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:50:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:33.664074 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:33.773190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e63d-8ef8-43e5-bc18-f7c6d9102455 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:33.808849 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:33.809759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:33.810039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:33.810312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] PATH_INFO: `/v3/users/fa837d5b5d9d449890a789f0861c602e` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:33.824134 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=fa837d5b5d9d449890a789f0861c602e)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:33.825640 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:33.827283 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 576/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:33 2022] GET /identity/v3/users/fa837d5b5d9d449890a789f0861c602e => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:33.833784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:33.926945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:33.940730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17649659-abf0-4164-9e7c-8e271b2e0e35 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:33.977408 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:33.978070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:33.978332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:33.978608 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:34.055498 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:34.056892 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:34.155736 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:34.170511 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 607/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:50:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:50:34.177299 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:34.237231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:34.290960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288472a-9bd6-453c-b0eb-75ca45fa5cd2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:34.329979 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a45f54-cb94-4224-a51c-fa329a06c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:34.330774 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a45f54-cb94-4224-a51c-fa329a06c335 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:34.331047 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a45f54-cb94-4224-a51c-fa329a06c335 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:34.331311 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a45f54-cb94-4224-a51c-fa329a06c335 None admin] PATH_INFO: `/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:34.332630 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a45f54-cb94-4224-a51c-fa329a06c335 None admin] RBAC: Authorizing `identity:get_project(project_id=3ce6fbf0e205453f9ebd2b192ffa1c08)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:34.334098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a45f54-cb94-4224-a51c-fa329a06c335 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:34.335816 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 577/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:34 2022] GET /identity/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:34.346066 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:34.455477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cda286-7767-458c-860c-a167b444eee5 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:34.491713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:34.492366 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:34.492648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:34.492912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:34.493798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=3ce6fbf0e205453f9ebd2b192ffa1c08, user.id=fa837d5b5d9d449890a789f0861c602e) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:34.494114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:34.495611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72294db-dbf8-473a-bbd3-45d4d5e92f00 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:34.501282 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 608/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:50:34 2022] GET /identity/v3/role_assignments?user.id=fa837d5b5d9d449890a789f0861c602e&scope.project.id=3ce6fbf0e205453f9ebd2b192ffa1c08&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:36.679159 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339b39e-c02f-43a1-8982-3227423f2c4e None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:36.679674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339b39e-c02f-43a1-8982-3227423f2c4e None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:36.679674 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339b39e-c02f-43a1-8982-3227423f2c4e None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:36.681090 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 578/1186] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:36.687473 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:36.687920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:36.688039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:36.715552 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:36.717413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:36.772711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:36.796453 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 609/1187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:36.819477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339b39e-c02f-43a1-8982-3227423f2c4e None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:36.924277 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339b39e-c02f-43a1-8982-3227423f2c4e None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:36.943062 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:36.962757 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:36.963553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:36.963835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:36.964117 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:36.974939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:36.982772 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:36.984037 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:50:36.984999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 579/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:50:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:36.991739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:37.099671 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fdb4a-5a2f-4349-8b96-c197b71355a7 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:37.135190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:37.135850 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.136153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.136455 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.137202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:37.137537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:37.139071 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:37.147632 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 610/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:37.156325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:37.258991 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade46f6-9798-46a8-bb9d-256eebfa627c None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:37.294376 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:37.295250 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.295525 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.295795 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] PATH_INFO: `/v3/users/fa837d5b5d9d449890a789f0861c602e` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.297136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] RBAC: Authorizing `identity:get_user(user_id=fa837d5b5d9d449890a789f0861c602e)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:37.298611 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:37.300223 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 580/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:37 2022] GET /identity/v3/users/fa837d5b5d9d449890a789f0861c602e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:37.308755 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:37.356387 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:37.413016 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99c193-e915-452d-8c72-b258e85a26ab None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:37.453137 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d80b6d-dfe8-4abe-829c-b986047fa0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:37.454057 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d80b6d-dfe8-4abe-829c-b986047fa0d3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.454338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d80b6d-dfe8-4abe-829c-b986047fa0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.454610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d80b6d-dfe8-4abe-829c-b986047fa0d3 None admin] PATH_INFO: `/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.456034 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d80b6d-dfe8-4abe-829c-b986047fa0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=3ce6fbf0e205453f9ebd2b192ffa1c08)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:37.457544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d80b6d-dfe8-4abe-829c-b986047fa0d3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:37.459315 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 611/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:37 2022] GET /identity/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:37.465703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:37.512843 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:37.568987 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4d1f2-a978-41af-9dbc-961508b24087 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:37.613488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4ca7-6c9e-4d2f-8af2-d1244093c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:37.614844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4ca7-6c9e-4d2f-8af2-d1244093c61e None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:37.615225 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4ca7-6c9e-4d2f-8af2-d1244093c61e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:37.615589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4ca7-6c9e-4d2f-8af2-d1244093c61e None admin] PATH_INFO: `/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08/users/fa837d5b5d9d449890a789f0861c602e/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:37.618877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4ca7-6c9e-4d2f-8af2-d1244093c61e None admin] RBAC: Authorizing `identity:create_grant(project_id=3ce6fbf0e205453f9ebd2b192ffa1c08, user_id=fa837d5b5d9d449890a789f0861c602e, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:37.621024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4ca7-6c9e-4d2f-8af2-d1244093c61e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:37.635792 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 581/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:50:37 2022] PUT /identity/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08/users/fa837d5b5d9d449890a789f0861c602e/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:50:39.823775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f3042-f3e3-458d-91b1-6b09c001c67c None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:39.824379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f3042-f3e3-458d-91b1-6b09c001c67c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:39.824379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f3042-f3e3-458d-91b1-6b09c001c67c None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:39.825788 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 612/1193] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:50:39.832277 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:39.833085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:39.833300 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:39.858950 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:50:39.860846 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:50:39.935181 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 582/1194] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:50:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:50:39.970012 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f3042-f3e3-458d-91b1-6b09c001c67c None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.092082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f3042-f3e3-458d-91b1-6b09c001c67c None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.106980 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:40.131072 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.131876 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.132162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.132438 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.143321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:50:40.151444 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:50:40.152638 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:50:40.155420 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 613/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:50:40.161364 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.201498 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:40.272691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d786b0b-97f0-4e99-815c-6f74e4283be8 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.310977 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.312009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.312470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.312912 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.314201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:40.314689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:40.316915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:40.328358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 583/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:40.334436 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.442445 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9efb-e989-493c-a2d5-d9a602c06c83 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.481720 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.482537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.482816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.483089 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] PATH_INFO: `/v3/users/fa837d5b5d9d449890a789f0861c602e` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.484344 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=fa837d5b5d9d449890a789f0861c602e)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:40.485911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:40.487566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 614/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/users/fa837d5b5d9d449890a789f0861c602e => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:40.496319 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.593801 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:40.600378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6394f-77db-432d-baf5-36b3e15c0b70 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.638241 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.638900 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.639178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.639451 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.711346 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:40.712814 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:40.814470 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:40.821869 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 584/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:50:40.831465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:40.879232 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:40.938447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76d487-a912-448c-8302-7f323b1a4dc1 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:40.973407 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b9cfb-f52e-4383-8b94-3649834fe433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:40.974279 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b9cfb-f52e-4383-8b94-3649834fe433 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:40.974559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b9cfb-f52e-4383-8b94-3649834fe433 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:40.974833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b9cfb-f52e-4383-8b94-3649834fe433 None admin] PATH_INFO: `/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:40.976140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b9cfb-f52e-4383-8b94-3649834fe433 None admin] RBAC: Authorizing `identity:get_project(project_id=3ce6fbf0e205453f9ebd2b192ffa1c08)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:40.977616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b9cfb-f52e-4383-8b94-3649834fe433 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:40.979336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 615/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/projects/3ce6fbf0e205453f9ebd2b192ffa1c08 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:50:40.986131 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:50:41.089800 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df9dc-cde3-45e9-a6ee-14cc07504f4e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:50:41.136949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:50:41.137629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:50:41.137911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:50:41.138187 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:50:41.139021 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=3ce6fbf0e205453f9ebd2b192ffa1c08, user.id=fa837d5b5d9d449890a789f0861c602e) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:50:41.139333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:50:41.140771 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48977a10-dad9-456d-80ef-13ad499248cc None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:50:41.146997 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 585/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:50:40 2022] GET /identity/v3/role_assignments?user.id=fa837d5b5d9d449890a789f0861c602e&scope.project.id=3ce6fbf0e205453f9ebd2b192ffa1c08&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:50:50.968284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=46 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:50:51.144024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=70 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.160060 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e6187-db7b-481b-9709-becc6a100f6e None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.160717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e6187-db7b-481b-9709-becc6a100f6e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.160717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e6187-db7b-481b-9709-becc6a100f6e None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.162377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 616/1201] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:51:39.168457 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.169086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.169086 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.189172 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:39.190485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:39.277343 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 586/1202] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:51:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:51:39.301847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e6187-db7b-481b-9709-becc6a100f6e None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.417743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e6187-db7b-481b-9709-becc6a100f6e None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.457570 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.458390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.458665 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.458943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.469616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.471069 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:39.489588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:39.490674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:39.491749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:39.491749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:39.491749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:39.491749 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:39.493637 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 617/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:39.496943 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.605870 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cfc96-eb8a-4ffe-8a2b-11e6ceb8fe11 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.642347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.650253 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.650945 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.651222 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.651490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.652225 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:39.652529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.654011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea5bf4-f375-4c36-baf5-67cc32d8c82c None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.669641 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 587/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:39.672389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:39.729132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:39.781330 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38594d3b-8d78-4164-9ec1-6e83f5049a07 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:39.815803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388575fa-a142-4d3c-b9d8-10b869f95fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:39.816486 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388575fa-a142-4d3c-b9d8-10b869f95fdf None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:39.816775 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388575fa-a142-4d3c-b9d8-10b869f95fdf None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:39.817042 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388575fa-a142-4d3c-b9d8-10b869f95fdf None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:39.817784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388575fa-a142-4d3c-b9d8-10b869f95fdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:39.819236 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388575fa-a142-4d3c-b9d8-10b869f95fdf None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:39.842519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 618/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 14:51:39 2022] POST /identity/v3/users => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:51:42.474363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277bbf4-ad75-4026-8d38-1efbecf1131a None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.475009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277bbf4-ad75-4026-8d38-1efbecf1131a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.475009 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277bbf4-ad75-4026-8d38-1efbecf1131a None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.476302 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 588/1206] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:42.482743 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.483214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.483325 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.502943 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:42.504432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:42.590455 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 619/1207] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:51:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:51:42.621951 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277bbf4-ad75-4026-8d38-1efbecf1131a None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.763339 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277bbf4-ad75-4026-8d38-1efbecf1131a None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.805564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.806394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.806746 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.807003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.818256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:42.828818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:42.829949 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:42.831491 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 589/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:42.839360 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:42.883416 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:42.952806 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cb9d8-0b69-4ea2-bb0f-653d8eaac315 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:42.995624 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:42.996347 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:42.996582 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:42.996855 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:42.997648 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:42.997956 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:42.999405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.009269 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 620/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:51:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:43.018297 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.076127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.139272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1cbe6-df47-4731-9cdd-012c1e171929 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.177801 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.178720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.179071 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.179421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.191734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.193382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:43.216616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:43.217993 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:43.219023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:43.219023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:43.219023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:43.219023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:43.219023 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 590/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:43.224716 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.342613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.345839 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9040260-c64a-431e-aa99-3e94208dac51 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.381945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.382672 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.383021 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.383368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.384174 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:43.384550 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.388784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.402709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 621/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:43.410957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.536421 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf4ced-fac5-4a08-9b98-0cc72f2e0a27 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.539165 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:43.586562 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.587838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.588337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.588773 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] PATH_INFO: `/v3/users/placement` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.598158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.599655 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:43.618216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:43.620323 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:43.622488 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 591/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:43.631009 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.756267 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973325d0-a529-4031-a438-d2592bc1831e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.802080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.803331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.803620 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.803897 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:43.804802 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:43.805106 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:43.806623 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:43.825593 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 622/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:43.831348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:43.950228 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95da72-1fc7-411f-b060-2d09c9fadd5f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:43.997906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:43.998698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:43.998976 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:43.999274 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.035742 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.081089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.082675 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.220229 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 592/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:51:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:51:44.226297 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.246547 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.337922 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba691e4-3dbf-488d-9602-934624ed01b7 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.374433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.375254 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.375534 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.375993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.386816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.388277 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:44.406383 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:44.407580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:44.408921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:44.408921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:44.408921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:44.408921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:44.408921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 623/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:44.413552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.490894 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.540203 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520d060-cc0f-4e19-b9bc-465d1e7425be None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.578077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.579105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.579581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.580104 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.581010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:44.581350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.582735 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.595795 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 593/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:44.601953 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.690616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.730751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c284-a4ea-451c-96f4-12de8bb13c2f None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.775600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.776340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.776619 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.776885 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.784699 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.786196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:51:44.801206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:44.802541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:44.803574 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 624/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:44.810638 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:44.931510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cc672-93a0-42fd-991c-cb1774402cb7 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:44.955770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:44.969483 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:44.970231 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:44.970539 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:44.970786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:44.971639 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:44.971909 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:44.973372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777e6c-7e47-4776-b281-7a7323fddb68 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:44.986207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 594/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:44.992751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:45.101462 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0eadf5-8d8a-4120-b402-5b601709c2b6 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:45.133825 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:45.143186 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:45.143818 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:45.144123 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:45.144388 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:45.145686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=2e72f8cfc30f4424bbad9445a38ef434) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:45.146011 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:45.147472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e569f2-e1a2-44b4-ad6f-afc4b57c6ee0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:45.153971 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 625/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:51:44 2022] GET /identity/v3/role_assignments?user.id=2e72f8cfc30f4424bbad9445a38ef434&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.401533 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0552a4-3b39-4c77-a083-77ce0d4e47b6 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.402114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0552a4-3b39-4c77-a083-77ce0d4e47b6 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.402114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0552a4-3b39-4c77-a083-77ce0d4e47b6 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.403530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 595/1220] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:47.410379 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.410893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.410893 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.429263 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:47.430499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:47.503018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 626/1221] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:51:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:51:47.526087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0552a4-3b39-4c77-a083-77ce0d4e47b6 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.634979 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0552a4-3b39-4c77-a083-77ce0d4e47b6 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.675093 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.675844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.676136 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.676385 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.687476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:47.695493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:47.696911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:47.697928 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 596/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:47.703129 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.814235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b32df-038e-4fa4-a262-38c6f681c9c5 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:47.840468 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.854413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:47.855403 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:47.855734 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:47.856055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:47.856852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] RBAC: Adding query filter params (name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:47.857162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:47.858580 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:47.868476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 627/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:47.874304 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:47.923720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:47.985552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366dd41-8f1f-40e7-8092-39e4d26f83cf None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.024229 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.025269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.025656 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.026007 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.038844 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.040358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:48.062842 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:48.064327 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:48.065489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:48.065489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:48.065489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:48.065489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:48.065489 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 597/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:48.072871 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.190447 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1f992-3103-41d8-ab60-29d5ceecdf09 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.228449 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.229156 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.229469 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.229742 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.230470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:48.230800 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.232157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.244428 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 628/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.250975 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.363796 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.371501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2880f-3931-4c3c-8413-caed1c289f50 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.409564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.410392 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.410696 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.410939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.419838 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.421325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:48.437150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:48.438453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:48.439687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 598/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:48.445377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.497950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.563640 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b766f-9367-4660-9043-4a30a8a95c15 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.604400 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.605124 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.605455 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.605777 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.606631 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:48.606906 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.608346 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.629941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 629/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.637418 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.748217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d3e3-7577-4dde-941a-89b952c5c2ed None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.787734 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.788584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.788910 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.789216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.801576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.803050 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:48.820410 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:48.821666 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:48.822706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:48.822706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:48.822706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:48.822706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:48.822706 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 599/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:48.827607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:48.930234 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a501ce-c5be-43e3-a8ea-e73821199dfa None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:48.932666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:48.968333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:48.969045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:48.969336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:48.969609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:48.970334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:48.970639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:48.972024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:48.984151 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 630/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:48.990114 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.000104 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:49.097135 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456d85f-925e-4232-98d3-41df57c99795 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.133877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.134687 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.134957 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.135230 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.142536 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.144105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:51:49.160487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:49.161713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:49.162760 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 600/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:51:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:49.168082 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.276363 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a674446-f3a8-4bab-9984-f04076a0db72 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.311798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.312549 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.312863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.313203 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.314200 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:49.314578 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.316138 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b34f8-5da0-4ccc-a4b5-4e6726224ece None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.330796 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 631/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:49.339829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:49.414701 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:49.451919 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d7ae5-33ea-481a-8881-78eb1a266a21 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:49.486622 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1e758-70c0-428d-b90d-55b2648d8478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:49.487537 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1e758-70c0-428d-b90d-55b2648d8478 None admin] REQUEST_METHOD: `PUT` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:49.487793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1e758-70c0-428d-b90d-55b2648d8478 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:49.488059 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1e758-70c0-428d-b90d-55b2648d8478 None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/2e72f8cfc30f4424bbad9445a38ef434/roles/931260cf51714e41884b328407aed54f` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:49.503597 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1e758-70c0-428d-b90d-55b2648d8478 None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=2e72f8cfc30f4424bbad9445a38ef434, role_id=931260cf51714e41884b328407aed54f)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:49.505084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1e758-70c0-428d-b90d-55b2648d8478 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:49.517863 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 601/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:51:49 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/2e72f8cfc30f4424bbad9445a38ef434/roles/931260cf51714e41884b328407aed54f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:51:51.833219 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb896db-bfb8-492f-bbe1-fe5bed9e4d25 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:51.833914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb896db-bfb8-492f-bbe1-fe5bed9e4d25 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:51.833914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb896db-bfb8-492f-bbe1-fe5bed9e4d25 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:51.835224 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 632/1233] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:51.840813 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:51.841374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:51.841574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:51.859216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:51.860419 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:51.930394 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 602/1234] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:51:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:51:51.954202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb896db-bfb8-492f-bbe1-fe5bed9e4d25 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:51.979861 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.065832 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb896db-bfb8-492f-bbe1-fe5bed9e4d25 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.104559 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.105472 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.105746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.106014 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] PATH_INFO: `/v3/roles/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.117026 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:52.126104 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:52.132376 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:52.133653 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 633/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 14:51:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:52.134482 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.184854 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.246577 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b236177-ac3e-4c81-b890-dc3477d29634 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.283583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.284315 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.284598 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.284872 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.285671 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] RBAC: Adding query filter params (name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:52.286002 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:52.287382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:52.295642 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 603/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:52.301865 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.401986 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.419040 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495650f-be02-4de8-9a32-4c39226a20df None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.455942 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.456921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.457331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.457699 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.470109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:52.471683 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:52.490220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:52.491377 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:52.492572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:52.492572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:52.492572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:52.492572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:52.493463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 634/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:52.498105 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.627906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04c4e4-1da2-46d4-a11e-15b59067583f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.643301 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.676318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.677438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.677897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.678171 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.679036 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:52.679340 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:52.680885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:52.699959 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 604/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:52.701562 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.811858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4222fe-dd64-4e7a-acaf-46eed620b56e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:52.848798 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:52.849656 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:52.849936 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:52.850204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] PATH_INFO: `/v3/users/placement` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:52.853267 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:52.859224 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:52.860939 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:52.875554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:52.876863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:52.878217 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:52.881433 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 635/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:52.884164 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:52.987513 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280e379-1b46-4a89-86d0-f5bec49ac0c8 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.027595 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.028279 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.028557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.028825 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.029717 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:53.030028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.031558 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.050485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 605/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:51:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:53.056564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.171709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccd92c-ca64-452e-896d-6a30baa52b18 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.221202 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.221889 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.222176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.222452 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.294633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.302277 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.303941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.418392 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 636/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:51:53.424728 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.460019 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.542647 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a0e6-c41f-4023-8cbc-bedfb0513332 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.592636 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.593580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.593883 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.594153 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.605614 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.607447 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:53.627190 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:53.628565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:53.630084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:53.630084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:53.630084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:53.630084 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:53.633797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 606/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:53.634880 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.743951 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.751114 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ecc63-d7cc-4814-aeb8-885a8fa0b81d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.793525 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.794270 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.794649 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.795179 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.795917 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:53.796220 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.797750 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:53.812370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 637/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:53.818501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:53.910718 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:53.940100 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da9d90-cafd-470a-9b4d-3d3f2557b626 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:53.979903 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:53.980782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:53.981097 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:53.981387 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:53.988523 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:53.990044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:51:54.007397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:54.008683 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:54.009916 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 607/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:51:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:54.017331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:54.137047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4503a-f439-4989-aaa5-e40906a80bda None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:54.182805 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:54.183688 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.183995 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.184265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.185146 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:54.185541 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:54.187476 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385b748-8a4b-4de9-8881-26474d987096 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.203499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 638/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:51:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:54.209837 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:54.321486 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd762af3-2133-46eb-9f35-fbff13e751a2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:54.366266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:54.367024 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:54.367251 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:54.367476 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:54.368318 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] RBAC: Adding query filter params (role.id=931260cf51714e41884b328407aed54f, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=2e72f8cfc30f4424bbad9445a38ef434) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:54.368632 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:54.370201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd5c3d-918a-43bc-8a2b-e8fdebebbbdb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:54.376580 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 608/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:51:54 2022] GET /identity/v3/role_assignments?user.id=2e72f8cfc30f4424bbad9445a38ef434&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=931260cf51714e41884b328407aed54f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:56.713226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78a05-815a-49ff-aacb-545ff1a98ef6 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.713836 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78a05-815a-49ff-aacb-545ff1a98ef6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.713836 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78a05-815a-49ff-aacb-545ff1a98ef6 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.715227 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 639/1247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:51:56.721163 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.721672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.721777 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:56.727094 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:56.739915 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:51:56.741298 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:51:56.824494 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 609/1248] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:51:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:51:56.850958 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78a05-815a-49ff-aacb-545ff1a98ef6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:56.870459 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:56.960507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78a05-815a-49ff-aacb-545ff1a98ef6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:56.998065 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:56.998879 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:56.999137 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:56.999432 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.010395 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:51:57.018502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.019611 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:57.021554 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 640/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:51:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:57.026514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.145270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458b3de-5e37-41fd-832f-dde5b1745016 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.181634 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.187888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.188578 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.188865 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.189205 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.189911 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.190217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.191606 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.200281 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 610/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:57.206229 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.292712 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.318539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46163f-bce3-44d7-990a-c1b0d491d2c0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.364564 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.365487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.365771 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.366045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.378545 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.380087 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:57.399691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.405900 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.407226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:57.407226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:57.407226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:57.407226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:57.407226 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 641/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:57.407610 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.526965 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227e09-299c-4e5b-b841-a095a4076fdf None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.568256 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.569001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.569404 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.569780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.570621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.571038 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.572462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.587961 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 611/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:51:57.593882 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.704507 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc760b6e-71dd-4aa2-9384-772f14e460b3 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.737771 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.742558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.743429 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.743723 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.744075 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] PATH_INFO: `/v3/users/placement` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.753499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.755044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:51:57.770201 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:57.771833 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:51:57.773566 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 642/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:51:57.780011 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:57.846458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:57.897617 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bcd9d-7da8-4a1c-a7e3-d976e4cfe904 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:57.936896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:57.937740 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:57.938132 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:57.938405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:57.939405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:57.939708 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:57.941347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:57.961985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 612/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:57.968674 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.082261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219dd67-9c5e-4986-927d-f0cbad54326d None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.124116 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.124853 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.125135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.125470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.190738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.206025 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.207506 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.319209 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 643/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:51:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:51:58.325503 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.447888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d7aa-99a3-4a11-a252-fb107cb122aa None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.462835 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.486930 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.487830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.488141 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.488458 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.499178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.500631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:51:58.518738 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:58.520389 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:58.521809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:51:58.521809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:51:58.521809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:51:58.521809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:58.521809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 613/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:58.526262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.628539 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:58.643141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d098-df29-4567-bac9-728b91e00ffc None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.683505 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.684184 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.684495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.684746 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.685545 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:58.685816 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.687268 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.699350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 644/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:58.705085 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:58.827779 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ca3a5-dd62-46bd-a036-2a54f659ed1e None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:58.878348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:58.879178 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:58.879462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:58.879771 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:58.887337 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:58.889098 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:51:58.906308 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:51:58.907493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:51:58.908698 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 614/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 14:51:58.914325 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.027980 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688a2f6-a3dc-41c5-80f8-ad51e0d22147 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.066305 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.067064 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.067348 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.067622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] PATH_INFO: `/v3/projects` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.068533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:59.068747 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.070211 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae46c5-fae6-4c0e-9c27-e5eb09b6c18c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.084322 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 645/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:51:59.091361 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:51:59.119296 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:51:59.211504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e364a-4ba8-4855-91cb-30a44359e477 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:51:59.251412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:51:59.252087 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:51:59.252366 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:51:59.252642 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:51:59.253524 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=2e72f8cfc30f4424bbad9445a38ef434) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:51:59.253837 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:51:59.255290 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395dd69-d665-4b7a-b14a-42f21f7c27cd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:51:59.260918 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 615/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:51:59 2022] GET /identity/v3/role_assignments?user.id=2e72f8cfc30f4424bbad9445a38ef434&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=d450d657419649a6ae78b9e8578202ca => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:01.505835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ceb24-fdec-46d1-8edf-ece393986ef7 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.506435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ceb24-fdec-46d1-8edf-ece393986ef7 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.506435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ceb24-fdec-46d1-8edf-ece393986ef7 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.507911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 646/1261] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:01.516852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.517552 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.518014 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.540158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:01.541399 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:01.613285 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 616/1262] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:01.636568 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ceb24-fdec-46d1-8edf-ece393986ef7 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.648812 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.745139 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ceb24-fdec-46d1-8edf-ece393986ef7 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.782609 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.783417 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.783697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.783969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.795386 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:01.803164 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:01.804345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:01.805705 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 647/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:01.810789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:01.831971 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:01.918682 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c691d-acc8-4733-b692-8b703bde1255 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:01.952690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:01.953383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:01.953665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:01.953939 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] PATH_INFO: `/v3/roles` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:01.954689 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:01.954998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:01.956374 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:01.964574 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 617/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:01.970909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.040765 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.075973 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0899c8-a052-4966-b362-92e686d345e8 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.110697 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.111514 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.111783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.112046 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.123273 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.124711 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:02.142196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:02.143323 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:02.144544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:02.144544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:02.144544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:02.144544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:02.144544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 648/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:02.149663 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.229834 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.257102 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfaa55-8649-452b-9276-d124624818ca None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.294603 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.295193 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.295477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.295747 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.296481 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:02.296793 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.298182 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.311365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 618/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:02.319146 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.447477 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187abec-fe4f-441c-83c2-8f7762ff0841 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.467706 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.491024 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.492237 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.492607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.492977 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.501945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.503418 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:02.518084 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:02.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:02.521413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 649/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:02.525829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.636499 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a70c0-0003-4146-ae52-4b648f38e321 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.675321 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.675998 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.676293 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.676555 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] PATH_INFO: `/v3/users` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.677428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:02.677745 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.679201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.688217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:02.697697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 619/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:02.703826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.807744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac358b-b8c3-4e9b-8d50-842599124fe0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:02.845530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:02.846338 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:02.846613 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:02.846886 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:02.858204 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:02.859664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:02.877126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:02.878264 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:02.879830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:02.879830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:02.879830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:02.879830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:02.879830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 650/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:02.887291 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:02.992269 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8606e06-eb51-4305-a53f-64cea4331efb None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.030106 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.030778 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.031059 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.031333 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.032070 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:03.032382 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.033809 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.045598 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 620/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:03.052071 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.088615 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.164999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b366a89-4293-4c61-b8e8-938a888698f0 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.200586 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.201406 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.201690 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.201966 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.209205 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.210713 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:03.226157 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:03.227255 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:03.233500 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 651/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:03.234395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.301437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.362492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0250a-56f7-45bb-8b73-d83ddfed4951 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.397076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.397768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.398052 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.398325 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.399121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:03.399433 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.400874 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c36ac-b78e-4391-b36c-ebce40037699 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.413073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 621/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:03.421180 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:03.514727 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:03.529804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e78185-727c-4dec-adae-26abb80f9818 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:03.569709 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b40c7-91a9-4792-9229-13c11cbfd083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:03.570600 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b40c7-91a9-4792-9229-13c11cbfd083 None admin] REQUEST_METHOD: `PUT` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:03.570921 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b40c7-91a9-4792-9229-13c11cbfd083 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:03.571141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b40c7-91a9-4792-9229-13c11cbfd083 None admin] PATH_INFO: `/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/2e72f8cfc30f4424bbad9445a38ef434/roles/d450d657419649a6ae78b9e8578202ca` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:03.573085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b40c7-91a9-4792-9229-13c11cbfd083 None admin] RBAC: Authorizing `identity:create_grant(project_id=c85b36a764674590b7ebc90942bda1ab, user_id=2e72f8cfc30f4424bbad9445a38ef434, role_id=d450d657419649a6ae78b9e8578202ca)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:03.574605 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b40c7-91a9-4792-9229-13c11cbfd083 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:03.592309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 652/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 14:52:03 2022] PUT /identity/v3/projects/c85b36a764674590b7ebc90942bda1ab/users/2e72f8cfc30f4424bbad9445a38ef434/roles/d450d657419649a6ae78b9e8578202ca => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:52:05.801160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89998ed0-0d06-406f-a2e7-df47fabd7fa4 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.801770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89998ed0-0d06-406f-a2e7-df47fabd7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.801770 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89998ed0-0d06-406f-a2e7-df47fabd7fa4 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.803104 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 622/1274] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:05.812037 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:05.812814 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:05.812993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:05.834821 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:05.836030 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:05.908161 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 653/1275] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:05.931898 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89998ed0-0d06-406f-a2e7-df47fabd7fa4 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.055692 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89998ed0-0d06-406f-a2e7-df47fabd7fa4 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.073395 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.094514 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.095306 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.095581 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.095848 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.107122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 14:52:06.114690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.115887 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:06.117197 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 623/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 14:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.122287 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.208827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.231708 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e8ecf-3211-4b1f-b7d3-f806614efb1f None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.267072 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.267739 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.268019 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.268301 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] PATH_INFO: `/v3/roles` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.269023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.269357 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.271118 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.279368 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 654/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.285730 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.410158 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070075fc-5db2-4cfc-87dd-924f6bc1d2e4 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.449268 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.450076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.450437 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.450711 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.461677 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.463079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:06.480646 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.481830 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.483041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:06.483041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:06.483041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:06.483041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:06.483041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 624/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.487843 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.600502 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f88a3-7e83-4b0f-a95a-7aed71ac0e3e None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.638795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.639465 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.639740 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.640006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] PATH_INFO: `/v3/domains` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.640725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:06.641052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.642458 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.655350 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 655/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:06.661210 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.695583 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.768861 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d560b-85a0-4e8e-98f3-837d94b16fc0 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:06.807259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:06.808041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:06.808348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:06.808614 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:06.817273 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:06.818851 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 14:52:06.833362 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:06.834633 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:06.836111 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 625/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:06.842689 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:06.846830 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:06.964804 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d9e15-eaa6-408a-83c4-71def7a0ffe2 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.011335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.012018 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.012358 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.012732 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] PATH_INFO: `/v3/users` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.016039 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:07.016402 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.018167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.037652 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 656/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 14:52:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:07.050759 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.165270 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d42fe-4f42-4703-b938-2c0dd2913f96 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.200803 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.201477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.201751 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.202028 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.271533 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.272990 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.363843 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.379557 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 626/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 14:52:07.386141 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.467639 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.518670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc29fb-7ce8-4776-a466-4c7be7157e7b None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.560972 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.561826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.562109 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.562384 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.573214 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.574725 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 14:52:07.592482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.593633 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.594847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 14:52:07.594847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 14:52:07.594847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 14:52:07.594847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:07.594847 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 657/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.599609 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.705804 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861be99-85f1-4835-929c-c7779a69d6f2 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.741412 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.742077 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.742358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.742631 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] PATH_INFO: `/v3/domains` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.743393 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:07.743699 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.745075 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.756618 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 627/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:07.762499 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.868908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e4216-a92c-4124-8fc5-a7a180dc9455 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:07.883284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:07.905744 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:07.906474 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:07.906754 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:07.907023 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:07.914134 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:07.915588 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 14:52:07.930047 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application with Lock( May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:07.931194 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: ERROR keystone.server.flask.application May 19 14:52:07.932351 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 658/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:07.939291 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:07.991248 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:08.054407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf2fa9-6f5a-47d8-8eb5-cda4cad84b7f None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.089348 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.090010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.090288 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.090592 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.091407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:08.091713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.093147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e2d9-0c29-4b67-9f06-6ee4b41f8bd3 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.105284 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 628/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 14:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:08.112280 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:08.229761 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53c9d1-0acb-4d5b-9f6e-7712c02c4e0f None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:08.266642 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:08.267289 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:08.267572 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:08.267848 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:08.268682 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] RBAC: Adding query filter params (role.id=d450d657419649a6ae78b9e8578202ca, scope.project.id=c85b36a764674590b7ebc90942bda1ab, user.id=2e72f8cfc30f4424bbad9445a38ef434) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:08.268998 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:08.270846 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fc42c-ec26-4dc4-8469-839c6cebdd43 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:08.276932 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 659/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 14:52:08 2022] GET /identity/v3/role_assignments?user.id=2e72f8cfc30f4424bbad9445a38ef434&scope.project.id=c85b36a764674590b7ebc90942bda1ab&role.id=d450d657419649a6ae78b9e8578202ca => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:10.501886 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb421c8b-25f4-40fc-9abd-322814ab8ac1 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.502507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb421c8b-25f4-40fc-9abd-322814ab8ac1 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.502507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb421c8b-25f4-40fc-9abd-322814ab8ac1 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.503811 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 629/1288] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:10.510295 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.511003 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.511183 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.535261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:10.536482 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:10.569080 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:10.608872 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 660/1289] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:52:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:10.631517 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb421c8b-25f4-40fc-9abd-322814ab8ac1 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.738017 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb421c8b-25f4-40fc-9abd-322814ab8ac1 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.749690 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:10.777039 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.777818 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.778127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.778428 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] PATH_INFO: `/v3/services/placement` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.779082 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.780466 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:10.792858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:10.794521 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:10.795768 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 630/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:10.800673 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:10.911284 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c06d2-f413-4a24-b98b-d53ee689562d None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:10.954361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:10.955031 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:10.955316 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:10.955577 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:10.956290 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] RBAC: Adding query filter params (name=placement) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:10.956601 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:10.958029 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56ab1b-00d0-4095-94d8-abe15ca8d473 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:10.963113 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 661/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:52:10.969207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:11.078665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a487180-dd67-4fc1-97b0-6f389a50e0fd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:11.117697 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b169384-8858-4527-8450-018d15a702fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:11.118365 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b169384-8858-4527-8450-018d15a702fb None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:11.118621 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b169384-8858-4527-8450-018d15a702fb None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:11.118924 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b169384-8858-4527-8450-018d15a702fb None admin] PATH_INFO: `/v3/services` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:11.119634 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b169384-8858-4527-8450-018d15a702fb None admin] RBAC: Adding query filter params (type=placement) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:11.119940 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b169384-8858-4527-8450-018d15a702fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:11.121357 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b169384-8858-4527-8450-018d15a702fb None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:11.126110 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 631/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:52:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:13.311718 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e1b0d-0d20-4d17-9ff1-5e2c6abcc025 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.312335 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e1b0d-0d20-4d17-9ff1-5e2c6abcc025 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.312692 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e1b0d-0d20-4d17-9ff1-5e2c6abcc025 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.315346 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 662/1293] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:52:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:13.324150 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e3ce6-869a-44f4-b042-6280fca8abfc None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.324584 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e3ce6-869a-44f4-b042-6280fca8abfc None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.324691 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e3ce6-869a-44f4-b042-6280fca8abfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.344564 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-7c1e3ce6-869a-44f4-b042-6280fca8abfc None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:13.346244 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-7c1e3ce6-869a-44f4-b042-6280fca8abfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:13.390532 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.434010 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 632/1294] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:13.457435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e1b0d-0d20-4d17-9ff1-5e2c6abcc025 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:13.493149 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:13.565031 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e1b0d-0d20-4d17-9ff1-5e2c6abcc025 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:13.598643 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e5967-26c2-46d0-8e1e-0a36b4b2dd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:13.599327 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e5967-26c2-46d0-8e1e-0a36b4b2dd89 None admin] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:13.599610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e5967-26c2-46d0-8e1e-0a36b4b2dd89 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:13.599881 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e5967-26c2-46d0-8e1e-0a36b4b2dd89 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:13.600604 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e5967-26c2-46d0-8e1e-0a36b4b2dd89 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:13.602045 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e5967-26c2-46d0-8e1e-0a36b4b2dd89 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:13.616112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 663/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 14:52:13 2022] POST /identity/v3/services => generated 246 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:15.859417 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b017cc-acd9-4424-8a76-53c25721082f None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.860038 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b017cc-acd9-4424-8a76-53c25721082f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.860038 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b017cc-acd9-4424-8a76-53c25721082f None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.861383 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 633/1296] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:15.867733 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:15.868463 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:15.868664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:15.888265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:15.889509 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:15.966606 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 664/1297] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:52:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 14:52:15.989048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b017cc-acd9-4424-8a76-53c25721082f None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.100341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b017cc-acd9-4424-8a76-53c25721082f None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.134981 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.135775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.136053 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.136322 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] PATH_INFO: `/v3/services/placement` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.136985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.138427 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:16.150780 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:16.152042 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:16.153088 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 634/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:52:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:16.160370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.182540 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.266467 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd1ff-9e90-4e86-8c76-4d1f2de1ecd6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.302610 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.303272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.303557 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.303831 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.304545 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] RBAC: Adding query filter params (name=placement) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.304877 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.306272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.312222 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 665/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.323672 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.359227 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.436856 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfd5c3-2a70-4b08-80eb-8731d7f52421 None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.477485 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.478155 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.478439 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.478712 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.479519 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] RBAC: Adding query filter params (interface=public, service_id=a63cd3e93b9541378acb81a18c70649e, region_id=RegionOne) {{(pid=213542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:16.479831 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.481221 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379621-ade0-46ff-bb38-cb270323f5d6 None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.486504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 635/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/endpoints?service_id=a63cd3e93b9541378acb81a18c70649e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:16.495496 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:16.580105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:16.603933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2fd43-a882-49f2-a1f6-12efcd0f7124 None admin] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:16.642517 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a45da-837e-4b72-a286-b6d272b3da3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:16.643532 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a45da-837e-4b72-a286-b6d272b3da3c None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:16.643905 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a45da-837e-4b72-a286-b6d272b3da3c None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:16.645070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a45da-837e-4b72-a286-b6d272b3da3c None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:16.645206 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a45da-837e-4b72-a286-b6d272b3da3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:16.646993 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a45da-837e-4b72-a286-b6d272b3da3c None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:16.656603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 666/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 14:52:16 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:52:18.881018 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f322e-0067-44e3-bbf2-e8776f8feffb None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.881985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f322e-0067-44e3-bbf2-e8776f8feffb None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.881985 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f322e-0067-44e3-bbf2-e8776f8feffb None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.885726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 636/1302] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:52:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:52:18.890835 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:18.891538 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:18.891687 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:18.913719 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:52:18.914914 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:52:18.997285 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 667/1303] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:52:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 14:52:19.021424 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f322e-0067-44e3-bbf2-e8776f8feffb None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.128167 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.137375 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f322e-0067-44e3-bbf2-e8776f8feffb None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.178339 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.179079 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.179361 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.179634 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] PATH_INFO: `/v3/services/placement` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.180291 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.181707 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 14:52:19.194407 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 14:52:19.195629 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.get_or_create( May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application with Lock( May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self._enter() May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application return self.creator() May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application created_value = creator( May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: ERROR keystone.server.flask.application May 19 14:52:19.196775 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 637/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 14:52:19.202812 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.281704 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:19.306869 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abddd0e-e26f-4fbe-97d9-6adc8dd5c1aa None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.348333 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.349001 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.349287 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.349583 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] PATH_INFO: `/v3/services` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.350286 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=213541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:52:19.350592 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.351969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652b430-76ba-4b48-bb5f-6dc7877a12a5 None admin] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.357624 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 668/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 14:52:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:52:19.363829 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:52:19.485358 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133259c7-4d66-4a44-928d-fb7d1136eddd None admin] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:52:19.524017 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70086eb2-9bc6-4189-98a7-0b8c448c647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7af2cbac96a492e94720a0ed454d0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:52:19.525041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70086eb2-9bc6-4189-98a7-0b8c448c647f None admin] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:52:19.525504 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70086eb2-9bc6-4189-98a7-0b8c448c647f None admin] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:52:19.525901 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70086eb2-9bc6-4189-98a7-0b8c448c647f None admin] PATH_INFO: `/v3/endpoints` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:52:19.527173 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70086eb2-9bc6-4189-98a7-0b8c448c647f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:52:19.528888 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70086eb2-9bc6-4189-98a7-0b8c448c647f None admin] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:52:19.555628 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 638/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 14:52:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:52:29.354228 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=30 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:52:29.547570 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:25.632753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b359535-a1e3-4fa2-97d9-fa98f223e5d9 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:25.633371 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b359535-a1e3-4fa2-97d9-fa98f223e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:25.633371 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b359535-a1e3-4fa2-97d9-fa98f223e5d9 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:25.635044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 669/1307] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 14:53:25.640894 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7ed2c-7f47-4aad-a49c-e0115779c8d0 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:25.641420 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7ed2c-7f47-4aad-a49c-e0115779c8d0 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:25.641506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7ed2c-7f47-4aad-a49c-e0115779c8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:25.785789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-30e7ed2c-7f47-4aad-a49c-e0115779c8d0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:25.787031 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-30e7ed2c-7f47-4aad-a49c-e0115779c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:25.905465 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 639/1308] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:25.966783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6fbe9-130e-411d-b708-55fecda71bb6 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:25.967556 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6fbe9-130e-411d-b708-55fecda71bb6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:25.968099 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6fbe9-130e-411d-b708-55fecda71bb6 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:25.969800 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 670/1309] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 14:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:25.977125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c732038-c2f0-4625-95ad-dcfa36577928 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:25.977507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c732038-c2f0-4625-95ad-dcfa36577928 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:25.977731 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c732038-c2f0-4625-95ad-dcfa36577928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:26.088426 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-3c732038-c2f0-4625-95ad-dcfa36577928 None None] MFA Rules not processed for user `a9976d68323c4f2b9f33554902ee2e85`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:26.089776 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-3c732038-c2f0-4625-95ad-dcfa36577928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:26.112216 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:26.240312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 640/1310] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:53:26.250235 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6fbe9-130e-411d-b708-55fecda71bb6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:26.252007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-8aa6fbe9-130e-411d-b708-55fecda71bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:26.464105 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6fbe9-130e-411d-b708-55fecda71bb6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:26.575945 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:26.577055 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:26.577598 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:26.578090 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:26.580296 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:26.647254 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:26.709457 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:26.711380 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a4261-9d7a-41d6-9a56-15912e16fdae service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:26.735920 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 671/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:29.334665 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba851d-6b54-4b26-be51-8f1a6c840734 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:29.335201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba851d-6b54-4b26-be51-8f1a6c840734 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:29.335201 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba851d-6b54-4b26-be51-8f1a6c840734 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:29.337254 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 641/1312] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:29.343933 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85be2-c142-4c0b-bf96-22768b5e44b0 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:29.344413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85be2-c142-4c0b-bf96-22768b5e44b0 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:29.344520 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85be2-c142-4c0b-bf96-22768b5e44b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:29.443324 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-35d85be2-c142-4c0b-bf96-22768b5e44b0 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:29.444526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-35d85be2-c142-4c0b-bf96-22768b5e44b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:29.571899 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 672/1313] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:29.610518 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba851d-6b54-4b26-be51-8f1a6c840734 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:29.612307 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-c1ba851d-6b54-4b26-be51-8f1a6c840734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:29.723251 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba851d-6b54-4b26-be51-8f1a6c840734 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:29.797266 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:29.799860 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:29.800704 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:29.801160 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:29.801627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:29.803526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:29.917072 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:29.918795 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa74bf-676b-4f72-b868-e95035945a1e service neutron] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:29.953696 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 642/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:32.530924 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904366-e6f3-406f-9399-693318c9c9d0 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:32.531495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904366-e6f3-406f-9399-693318c9c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:32.531495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82904366-e6f3-406f-9399-693318c9c9d0 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:32.532866 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 673/1315] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:32.538533 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b75d4e-3b8c-4208-9776-bf5e009bdd6e None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:32.538978 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b75d4e-3b8c-4208-9776-bf5e009bdd6e None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:32.539076 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b75d4e-3b8c-4208-9776-bf5e009bdd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:32.611942 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f6b75d4e-3b8c-4208-9776-bf5e009bdd6e None None] MFA Rules not processed for user `497304e37c85429c8c3148acfed14451`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:32.613195 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f6b75d4e-3b8c-4208-9776-bf5e009bdd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:32.675501 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:32.723988 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 643/1316] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:53:32.779294 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2bb63-b8e8-49da-999a-fa7d459920d5 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:32.779584 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2bb63-b8e8-49da-999a-fa7d459920d5 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:32.779852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2bb63-b8e8-49da-999a-fa7d459920d5 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:32.781265 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 674/1317] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 14:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:32.788347 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312939f7-999b-4349-9d81-c9312145bac1 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:32.788727 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312939f7-999b-4349-9d81-c9312145bac1 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:32.789119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312939f7-999b-4349-9d81-c9312145bac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:32.878684 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-312939f7-999b-4349-9d81-c9312145bac1 None None] MFA Rules not processed for user `a9976d68323c4f2b9f33554902ee2e85`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:32.879947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-312939f7-999b-4349-9d81-c9312145bac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:32.971893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 644/1318] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:32.981079 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2bb63-b8e8-49da-999a-fa7d459920d5 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:32.982666 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-d1a2bb63-b8e8-49da-999a-fa7d459920d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:32.993396 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.092621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2bb63-b8e8-49da-999a-fa7d459920d5 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:33.176527 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:33.177176 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:33.177495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:33.177771 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:33.179446 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:33.255331 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:33.307519 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:33.308988 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7420136-c7f6-46a7-90cc-2f4384373c9c service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:33.327603 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 675/1319] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:36.502073 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5913a-ef0d-4d3a-b864-6a9dc47d25b5 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.502972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5913a-ef0d-4d3a-b864-6a9dc47d25b5 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.502972 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5913a-ef0d-4d3a-b864-6a9dc47d25b5 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.505260 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 645/1320] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:36.513042 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a9034-3902-4a6b-a337-074b04eea81d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.513544 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a9034-3902-4a6b-a337-074b04eea81d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.513655 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a9034-3902-4a6b-a337-074b04eea81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.583389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-561a9034-3902-4a6b-a337-074b04eea81d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:36.584622 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-561a9034-3902-4a6b-a337-074b04eea81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:36.694969 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 676/1321] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:53:36.736893 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5913a-ef0d-4d3a-b864-6a9dc47d25b5 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:36.738496 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-54b5913a-ef0d-4d3a-b864-6a9dc47d25b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:36.795005 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:36.849722 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5913a-ef0d-4d3a-b864-6a9dc47d25b5 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:36.929207 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:36.929897 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:36.930180 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:36.930453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:36.932174 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:37.035154 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:37.049641 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:37.051036 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80f8bd-0d2b-48db-b8cd-a776e8ddf898 service neutron] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:37.069438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 646/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:40.039642 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ea3ea-e16d-4b2f-b0ed-63a3d616a9be None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.040167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ea3ea-e16d-4b2f-b0ed-63a3d616a9be None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.040167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ea3ea-e16d-4b2f-b0ed-63a3d616a9be None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.041607 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 677/1323] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:40.047041 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810de31-74ce-4028-8182-08d2475810cc None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.047510 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810de31-74ce-4028-8182-08d2475810cc None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.047616 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810de31-74ce-4028-8182-08d2475810cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.113995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-0810de31-74ce-4028-8182-08d2475810cc None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:40.115217 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-0810de31-74ce-4028-8182-08d2475810cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:40.218820 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 647/1324] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:40.254698 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ea3ea-e16d-4b2f-b0ed-63a3d616a9be None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:40.256249 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-167ea3ea-e16d-4b2f-b0ed-63a3d616a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:40.277340 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:40.367885 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ea3ea-e16d-4b2f-b0ed-63a3d616a9be None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:40.445626 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:40.446262 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:40.446536 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:40.446803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:40.448491 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:40.522779 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:40.562390 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:40.563734 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456f62d-2a45-4bd3-9e4a-acb5b5df3459 service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:40.582537 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 678/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:42.857956 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee668c17-dd35-482b-806e-ce6f123126a7 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:42.858506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee668c17-dd35-482b-806e-ce6f123126a7 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:42.858506 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee668c17-dd35-482b-806e-ce6f123126a7 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:42.859875 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 648/1326] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:42.865370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d266cd-86da-44f4-b7ee-8a3b5057a6c9 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:42.865826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d266cd-86da-44f4-b7ee-8a3b5057a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:42.865935 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d266cd-86da-44f4-b7ee-8a3b5057a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:42.934135 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-53d266cd-86da-44f4-b7ee-8a3b5057a6c9 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:42.935370 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-53d266cd-86da-44f4-b7ee-8a3b5057a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:43.034857 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 679/1327] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:53:43.070709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee668c17-dd35-482b-806e-ce6f123126a7 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:43.072477 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-ee668c17-dd35-482b-806e-ce6f123126a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:43.138575 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:43.186615 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee668c17-dd35-482b-806e-ce6f123126a7 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:43.272227 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:43.272896 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:43.273183 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:43.273456 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:43.275147 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:43.387700 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:43.390125 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:43.391563 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa203c0-69ab-4d4e-86be-98723708e063 service neutron] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:43.410571 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 649/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:45.666808 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950d72f-a695-4314-b77a-24e8666092e6 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:45.667299 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950d72f-a695-4314-b77a-24e8666092e6 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:45.667752 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950d72f-a695-4314-b77a-24e8666092e6 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:45.669827 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 680/1329] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:45.677300 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3586c67-6b5a-4ccd-97fa-32109a4447d1 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:45.677783 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3586c67-6b5a-4ccd-97fa-32109a4447d1 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:45.677886 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3586c67-6b5a-4ccd-97fa-32109a4447d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:45.756703 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-b3586c67-6b5a-4ccd-97fa-32109a4447d1 None None] MFA Rules not processed for user `497304e37c85429c8c3148acfed14451`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:45.757999 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-b3586c67-6b5a-4ccd-97fa-32109a4447d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:45.867565 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 650/1330] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:45.906738 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950d72f-a695-4314-b77a-24e8666092e6 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:45.908291 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-d950d72f-a695-4314-b77a-24e8666092e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:45.941858 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:46.021443 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950d72f-a695-4314-b77a-24e8666092e6 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:46.101467 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:46.102132 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:46.102413 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:46.102686 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:46.104460 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:46.187753 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:46.227702 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:46.229423 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafde31c-7c4e-4bc1-a29f-df9fad9be9c1 service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:46.247533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 681/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:48.699947 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503e2cc-3d53-4eae-9a5c-d3afe82eef79 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:48.700819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503e2cc-3d53-4eae-9a5c-d3afe82eef79 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:48.700819 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503e2cc-3d53-4eae-9a5c-d3afe82eef79 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:48.703259 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 651/1332] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:48.709891 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f1d0f-8aa7-41e3-9abc-920ef93e972e None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:48.710617 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f1d0f-8aa7-41e3-9abc-920ef93e972e None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:48.710818 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f1d0f-8aa7-41e3-9abc-920ef93e972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:48.783941 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-d32f1d0f-8aa7-41e3-9abc-920ef93e972e None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:48.785175 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-d32f1d0f-8aa7-41e3-9abc-920ef93e972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:48.889309 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 682/1333] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:48.925840 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503e2cc-3d53-4eae-9a5c-d3afe82eef79 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:48.927413 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-3503e2cc-3d53-4eae-9a5c-d3afe82eef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:49.017425 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:49.054948 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503e2cc-3d53-4eae-9a5c-d3afe82eef79 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:49.132822 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:49.133516 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:49.133824 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:49.134092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:49.135852 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:49.249312 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:49.250702 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b6d58-9df3-4821-9927-e8aa48aa9786 service neutron] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:49.260922 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:49.269885 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 652/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:51.516495 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa7e2-b6c9-4cef-8fda-db558c248a2a None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.517487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa7e2-b6c9-4cef-8fda-db558c248a2a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.517487 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa7e2-b6c9-4cef-8fda-db558c248a2a None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.520067 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 683/1335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:51.528003 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d524e-cdc5-4120-96ed-2d90b2dae9a6 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.528468 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d524e-cdc5-4120-96ed-2d90b2dae9a6 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.528579 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d524e-cdc5-4120-96ed-2d90b2dae9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.596989 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-835d524e-cdc5-4120-96ed-2d90b2dae9a6 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:51.598224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-835d524e-cdc5-4120-96ed-2d90b2dae9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:51.699001 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 653/1336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:53:51.739389 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa7e2-b6c9-4cef-8fda-db558c248a2a None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:51.741005 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-00daa7e2-b6c9-4cef-8fda-db558c248a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:51.788906 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:51.862907 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa7e2-b6c9-4cef-8fda-db558c248a2a None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:51.940931 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:51.941616 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:51.941894 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:51.942162 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:51.943783 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:52.033826 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:52.064007 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:52.065458 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058331f-c074-498f-92ce-a186f39abad1 service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:52.083960 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 684/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:53:54.751350 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ce8a-9cc6-4d66-bd29-c5d314b5fb85 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.751946 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ce8a-9cc6-4d66-bd29-c5d314b5fb85 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.751946 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ce8a-9cc6-4d66-bd29-c5d314b5fb85 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.753363 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 654/1338] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:54.759789 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c94c41-a998-46dc-a3c4-9130538e43b2 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:54.760548 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c94c41-a998-46dc-a3c4-9130538e43b2 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:54.760730 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c94c41-a998-46dc-a3c4-9130538e43b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:54.834909 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-47c94c41-a998-46dc-a3c4-9130538e43b2 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:54.836136 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-47c94c41-a998-46dc-a3c4-9130538e43b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:54.941595 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 685/1339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:54.977119 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ce8a-9cc6-4d66-bd29-c5d314b5fb85 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:54.978749 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-1aa8ce8a-9cc6-4d66-bd29-c5d314b5fb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:55.063877 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:55.101276 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ce8a-9cc6-4d66-bd29-c5d314b5fb85 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:53:55.176944 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:53:55.177623 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:55.177901 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:55.178175 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:55.179801 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:55.301549 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:53:55.303761 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13432f3d-9545-4053-aea7-26ecec5f4b77 service neutron] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:53:55.330188 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 655/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:53:59.712881 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38efa-e03d-42cd-957c-726fe6f4577c None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:59.713492 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38efa-e03d-42cd-957c-726fe6f4577c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:59.713492 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38efa-e03d-42cd-957c-726fe6f4577c None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:59.714849 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 686/1341] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:53:59.720240 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6a07-e81e-4268-ae33-50b01ea4d8b8 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:53:59.720709 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6a07-e81e-4268-ae33-50b01ea4d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:53:59.720827 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6a07-e81e-4268-ae33-50b01ea4d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:53:59.727922 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=100 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:53:59.796272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-3f1d6a07-e81e-4268-ae33-50b01ea4d8b8 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:53:59.797462 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-3f1d6a07-e81e-4268-ae33-50b01ea4d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:53:59.909272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 656/1342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:53:59.948022 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38efa-e03d-42cd-957c-726fe6f4577c None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:53:59.950664 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-98d38efa-e03d-42cd-957c-726fe6f4577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:00.004345 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:00.052978 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38efa-e03d-42cd-957c-726fe6f4577c None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:00.057063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72094aaf-06de-44ec-8946-92ae1b4dd1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:00.057759 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72094aaf-06de-44ec-8946-92ae1b4dd1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:00.058044 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72094aaf-06de-44ec-8946-92ae1b4dd1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:00.058324 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72094aaf-06de-44ec-8946-92ae1b4dd1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:00.067992 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72094aaf-06de-44ec-8946-92ae1b4dd1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:00.069563 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72094aaf-06de-44ec-8946-92ae1b4dd1f3 service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:00.079312 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 687/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:54:02.870334 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d534ef0-c700-484a-b958-ab8a20fe6212 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:02.870906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d534ef0-c700-484a-b958-ab8a20fe6212 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:02.870906 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d534ef0-c700-484a-b958-ab8a20fe6212 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:02.872406 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 657/1344] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:02.877844 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08e682-487a-46d9-bbde-05287d1e6c0f None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:02.878280 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08e682-487a-46d9-bbde-05287d1e6c0f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:02.878512 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08e682-487a-46d9-bbde-05287d1e6c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:02.908318 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-ab08e682-487a-46d9-bbde-05287d1e6c0f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:02.909651 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-ab08e682-487a-46d9-bbde-05287d1e6c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:02.992631 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 688/1345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:54:03.030402 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d534ef0-c700-484a-b958-ab8a20fe6212 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:03.042492 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d534ef0-c700-484a-b958-ab8a20fe6212 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:03.046790 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329913-2dd7-4bae-8d33-c4eb9a78d376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:03.047431 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329913-2dd7-4bae-8d33-c4eb9a78d376 service neutron] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:03.047713 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329913-2dd7-4bae-8d33-c4eb9a78d376 service neutron] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:03.047984 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329913-2dd7-4bae-8d33-c4eb9a78d376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:03.057405 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13329913-2dd7-4bae-8d33-c4eb9a78d376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:03.058862 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13329913-2dd7-4bae-8d33-c4eb9a78d376 service neutron] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:03.068391 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 658/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:54:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:54:07.349158 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffb38e-bb40-4319-8089-01ae4bbb088f None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:07.349745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffb38e-bb40-4319-8089-01ae4bbb088f None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:07.349745 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffb38e-bb40-4319-8089-01ae4bbb088f None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:07.351090 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 689/1347] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:07.356531 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ccb10-cd1c-4c1e-a24c-394595719d0b None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:07.356990 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ccb10-cd1c-4c1e-a24c-394595719d0b None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:07.357092 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ccb10-cd1c-4c1e-a24c-394595719d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:07.375901 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-f02ccb10-cd1c-4c1e-a24c-394595719d0b None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:07.377118 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-f02ccb10-cd1c-4c1e-a24c-394595719d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:07.386601 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 659/1348] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:07.424803 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffb38e-bb40-4319-8089-01ae4bbb088f None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:07.426751 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-0cffb38e-bb40-4319-8089-01ae4bbb088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:07.441435 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffb38e-bb40-4319-8089-01ae4bbb088f None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:07.445196 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19467297-4fd3-4de7-be0c-911d2d647105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9976d68323c4f2b9f33554902ee2e85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:07.445883 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19467297-4fd3-4de7-be0c-911d2d647105 service neutron] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:07.446167 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19467297-4fd3-4de7-be0c-911d2d647105 service neutron] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:07.446442 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19467297-4fd3-4de7-be0c-911d2d647105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:07.456470 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19467297-4fd3-4de7-be0c-911d2d647105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:07.457950 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19467297-4fd3-4de7-be0c-911d2d647105 service neutron] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:07.467605 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 690/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:54:13.066797 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e39e-1c31-4377-80b6-75e17e193b25 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:13.067367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e39e-1c31-4377-80b6-75e17e193b25 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:13.067367 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e39e-1c31-4377-80b6-75e17e193b25 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:13.069109 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 660/1350] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:13.074533 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212791a-6c9c-49ab-ace2-e4fc1e67fb0c None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:13.074999 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212791a-6c9c-49ab-ace2-e4fc1e67fb0c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:13.075126 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212791a-6c9c-49ab-ace2-e4fc1e67fb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:13.158660 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-5212791a-6c9c-49ab-ace2-e4fc1e67fb0c None None] MFA Rules not processed for user `2e72f8cfc30f4424bbad9445a38ef434`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:13.160277 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-5212791a-6c9c-49ab-ace2-e4fc1e67fb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:13.196006 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=100 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:13.290621 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 691/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:54:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:17.370530 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=72 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:17.570487 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bba0c-fb51-41ed-9599-bec185106d66 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:17.570871 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bba0c-fb51-41ed-9599-bec185106d66 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:17.571089 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bba0c-fb51-41ed-9599-bec185106d66 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:17.572720 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 661/1352] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:17.578092 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99704409-b579-4bc1-9958-d48ad038978a None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:17.578594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99704409-b579-4bc1-9958-d48ad038978a None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:17.578678 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99704409-b579-4bc1-9958-d48ad038978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:17.597912 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-99704409-b579-4bc1-9958-d48ad038978a None None] MFA Rules not processed for user `2e72f8cfc30f4424bbad9445a38ef434`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:17.599120 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-99704409-b579-4bc1-9958-d48ad038978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:17.609085 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 692/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:54:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:19.149415 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8b58c-fbca-4cbc-8f48-c6e91ac107fd None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:19.149995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8b58c-fbca-4cbc-8f48-c6e91ac107fd None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:19.149995 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8b58c-fbca-4cbc-8f48-c6e91ac107fd None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:19.151490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 662/1354] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 14:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:19.156630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd43365-3ec1-4b11-b4ea-d91ecfff6256 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:19.157070 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd43365-3ec1-4b11-b4ea-d91ecfff6256 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:19.157179 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd43365-3ec1-4b11-b4ea-d91ecfff6256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:19.177163 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-bdd43365-3ec1-4b11-b4ea-d91ecfff6256 None None] MFA Rules not processed for user `2e72f8cfc30f4424bbad9445a38ef434`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:19.178378 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-bdd43365-3ec1-4b11-b4ea-d91ecfff6256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:19.187630 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 693/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 14:54:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:23.059589 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6074d6-3767-489f-9313-941479c9baa3 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:23.060127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6074d6-3767-489f-9313-941479c9baa3 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:23.060127 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6074d6-3767-489f-9313-941479c9baa3 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:23.061484 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 663/1356] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:23.067153 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec74daf-b822-4ad6-ab14-0ff7ccc90222 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:23.067594 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec74daf-b822-4ad6-ab14-0ff7ccc90222 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:23.067717 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec74daf-b822-4ad6-ab14-0ff7ccc90222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:23.086394 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-eec74daf-b822-4ad6-ab14-0ff7ccc90222 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:23.087620 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-eec74daf-b822-4ad6-ab14-0ff7ccc90222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:23.095769 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 694/1357] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:23.148490 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5874e4-269f-4379-bb13-20fa15793f93 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:23.148771 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5874e4-269f-4379-bb13-20fa15793f93 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:23.149044 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5874e4-269f-4379-bb13-20fa15793f93 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:23.150568 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 664/1358] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 14:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:23.159332 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9456a-eca2-4f29-8c6e-a67c1709d98b None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:23.160190 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9456a-eca2-4f29-8c6e-a67c1709d98b None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:23.160795 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9456a-eca2-4f29-8c6e-a67c1709d98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:23.234025 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-11a9456a-eca2-4f29-8c6e-a67c1709d98b None None] MFA Rules not processed for user `502c9194cb0348f185dd9a5262ad1d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:23.235272 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-11a9456a-eca2-4f29-8c6e-a67c1709d98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:23.324525 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 695/1359] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 14:54:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:23.333627 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5874e4-269f-4379-bb13-20fa15793f93 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:23.348782 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5874e4-269f-4379-bb13-20fa15793f93 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:23.354593 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b077b-5b5a-4c20-be56-ad520ddd5008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502c9194cb0348f185dd9a5262ad1d2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:23.355414 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b077b-5b5a-4c20-be56-ad520ddd5008 service nova] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:23.355847 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b077b-5b5a-4c20-be56-ad520ddd5008 service nova] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:23.356272 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b077b-5b5a-4c20-be56-ad520ddd5008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:23.365066 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b077b-5b5a-4c20-be56-ad520ddd5008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:23.366958 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b077b-5b5a-4c20-be56-ad520ddd5008 service nova] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:23.377438 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 665/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:54:25.799348 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302d43b-2ca7-4b7d-bde2-cfdfb7acc0e5 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:25.799911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302d43b-2ca7-4b7d-bde2-cfdfb7acc0e5 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:25.799911 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302d43b-2ca7-4b7d-bde2-cfdfb7acc0e5 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:25.801304 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 696/1361] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:25.806938 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fcd7f-55b6-4ebf-b319-4120d5aa4d2f None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:25.807379 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fcd7f-55b6-4ebf-b319-4120d5aa4d2f None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:25.807543 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fcd7f-55b6-4ebf-b319-4120d5aa4d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:25.826224 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-817fcd7f-55b6-4ebf-b319-4120d5aa4d2f None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:25.827453 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-817fcd7f-55b6-4ebf-b319-4120d5aa4d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:25.835726 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 666/1362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:25.880530 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302d43b-2ca7-4b7d-bde2-cfdfb7acc0e5 None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:25.895784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302d43b-2ca7-4b7d-bde2-cfdfb7acc0e5 None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:25.901728 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b372a6-649f-476c-a35b-6bfb9aff15eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502c9194cb0348f185dd9a5262ad1d2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:25.902405 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b372a6-649f-476c-a35b-6bfb9aff15eb service nova] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:25.902691 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b372a6-649f-476c-a35b-6bfb9aff15eb service nova] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:25.902965 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b372a6-649f-476c-a35b-6bfb9aff15eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:25.912140 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b372a6-649f-476c-a35b-6bfb9aff15eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:25.913784 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b372a6-649f-476c-a35b-6bfb9aff15eb service nova] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:25.923658 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 697/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:54:28.486493 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866aa31-ab16-4efe-9e56-f4e066267a15 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:28.487122 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866aa31-ab16-4efe-9e56-f4e066267a15 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:28.487397 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866aa31-ab16-4efe-9e56-f4e066267a15 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:28.488960 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 667/1364] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:28.494579 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea65f0f-1936-4811-b14c-a3021669004d None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:28.495063 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea65f0f-1936-4811-b14c-a3021669004d None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:28.495158 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea65f0f-1936-4811-b14c-a3021669004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:28.513670 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-dea65f0f-1936-4811-b14c-a3021669004d None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:28.514908 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-dea65f0f-1936-4811-b14c-a3021669004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:28.523597 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 698/1365] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:28.569576 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e5ccd-8fc9-42d2-93d3-651ab4c28bc9 None None] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:28.569859 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e5ccd-8fc9-42d2-93d3-651ab4c28bc9 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:28.570341 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e5ccd-8fc9-42d2-93d3-651ab4c28bc9 None None] PATH_INFO: `/` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:28.571799 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 668/1366] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 14:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:28.577490 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a846-dca6-41f1-8731-fc343bfea709 None None] REQUEST_METHOD: `POST` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:28.577983 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a846-dca6-41f1-8731-fc343bfea709 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:28.578334 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a846-dca6-41f1-8731-fc343bfea709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:28.597987 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.auth.core [None req-b0f6a846-dca6-41f1-8731-fc343bfea709 None None] MFA Rules not processed for user `502c9194cb0348f185dd9a5262ad1d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=213541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:28.599293 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.fernet_utils [None req-b0f6a846-dca6-41f1-8731-fc343bfea709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:28.616342 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 699/1367] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 14:54:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:28.617678 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e5ccd-8fc9-42d2-93d3-651ab4c28bc9 None None] Authenticating user token {{(pid=213542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:28.639826 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e5ccd-8fc9-42d2-93d3-651ab4c28bc9 None None] Validating token access rules against request {{(pid=213542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:28.647553 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa6ab4-42ec-4520-bc7f-4603adb20db1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502c9194cb0348f185dd9a5262ad1d2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:28.648526 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6ab4-42ec-4520-bc7f-4603adb20db1 service nova] REQUEST_METHOD: `GET` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:28.648858 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6ab4-42ec-4520-bc7f-4603adb20db1 service nova] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:28.649194 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6ab4-42ec-4520-bc7f-4603adb20db1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:28.658681 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa6ab4-42ec-4520-bc7f-4603adb20db1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:28.660372 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa6ab4-42ec-4520-bc7f-4603adb20db1 service nova] RBAC: Authorization granted {{(pid=213542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:28.670901 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 669/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:54:29.465526 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d8f63-0595-4f76-8728-599f148f3611 None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:29.465863 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d8f63-0595-4f76-8728-599f148f3611 None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:29.466321 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d8f63-0595-4f76-8728-599f148f3611 None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:29.467884 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 700/1369] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:29.473529 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00dcbe-3fef-4ebc-9421-bfbb596e78e3 None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:29.473786 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00dcbe-3fef-4ebc-9421-bfbb596e78e3 None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:29.474255 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00dcbe-3fef-4ebc-9421-bfbb596e78e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:29.492725 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-9b00dcbe-3fef-4ebc-9421-bfbb596e78e3 None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:29.494048 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-9b00dcbe-3fef-4ebc-9421-bfbb596e78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:29.513894 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 670/1370] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:30.984361 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba105d42-352c-40e9-b924-2bf23e9bb18c None None] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:30.984929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba105d42-352c-40e9-b924-2bf23e9bb18c None None] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:30.984929 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba105d42-352c-40e9-b924-2bf23e9bb18c None None] PATH_INFO: `/` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:30.986240 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 701/1371] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 14:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:54:30.991607 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e340079-da57-41e8-a658-5e80ef7b0e2a None None] REQUEST_METHOD: `POST` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:30.992036 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e340079-da57-41e8-a658-5e80ef7b0e2a None None] SCRIPT_NAME: `/identity` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:30.992209 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e340079-da57-41e8-a658-5e80ef7b0e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=213542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:31.012121 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.auth.core [None req-9e340079-da57-41e8-a658-5e80ef7b0e2a None None] MFA Rules not processed for user `f7af2cbac96a492e94720a0ed454d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=213542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:54:31.013378 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG keystone.common.fernet_utils [None req-9e340079-da57-41e8-a658-5e80ef7b0e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=213542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:54:31.022789 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: [pid: 213542|app: 0|req: 671/1372] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:54:31.067112 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba105d42-352c-40e9-b924-2bf23e9bb18c None None] Authenticating user token {{(pid=213541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:54:31.083261 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba105d42-352c-40e9-b924-2bf23e9bb18c None None] Validating token access rules against request {{(pid=213541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:54:31.089399 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8ad2-5472-4e81-8efc-ec6d2725200c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502c9194cb0348f185dd9a5262ad1d2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c85b36a764674590b7ebc90942bda1ab', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=213541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:54:31.090052 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8ad2-5472-4e81-8efc-ec6d2725200c service nova] REQUEST_METHOD: `GET` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:54:31.090336 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8ad2-5472-4e81-8efc-ec6d2725200c service nova] SCRIPT_NAME: `/identity` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:54:31.090614 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8ad2-5472-4e81-8efc-ec6d2725200c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=213541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:54:31.098967 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8ad2-5472-4e81-8efc-ec6d2725200c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:54:31.100558 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8ad2-5472-4e81-8efc-ec6d2725200c service nova] RBAC: Authorization granted {{(pid=213541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:54:31.110053 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: [pid: 213541|app: 0|req: 702/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 14:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:54:41.006535 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: DEBUG dbcounter [-] [213542] Writing DB stats keystone:SELECT=21 {{(pid=213542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:41.105852 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: DEBUG dbcounter [-] [213541] Writing DB stats keystone:SELECT=95 {{(pid=213541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:54:53.367479 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: Thu May 19 14:54:53 2022 - graceful shutdown triggered... May 19 14:54:53.367005 dv-cinder-842542-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 14:54:53.368139 dv-cinder-842542-1-iscsi devstack@keystone.service[213541]: Gracefully killing worker 1 (pid: 213541)... May 19 14:54:53.368507 dv-cinder-842542-1-iscsi devstack@keystone.service[213542]: Gracefully killing worker 2 (pid: 213542)... May 19 14:54:54.368157 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: worker 1 buried after 1 seconds May 19 14:54:54.368157 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: worker 2 buried after 1 seconds May 19 14:54:54.368157 dv-cinder-842542-1-iscsi devstack@keystone.service[213530]: goodbye to uWSGI. May 19 14:54:54.370170 dv-cinder-842542-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 14:54:54.370986 dv-cinder-842542-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 15:00:47.489405 dv-cinder-842542-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 15:00:47.498584 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 15:00:47.501363 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 15:00:47 2022] *** May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: nodename: dv-cinder-842542-1-iscsi May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: machine: x86_64 May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: clock source: unix May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: pcre jit disabled May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: detected number of CPU cores: 4 May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: current working directory: / May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: detected binary path: /usr/bin/uwsgi-core May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: your processes number limit is 47680 May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: your memory page size is 4096 bytes May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: detected max file descriptor number: 2048 May 19 15:00:47.501547 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: lock engine: pthread robust mutexes May 19 15:00:47.502489 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: thunder lock: enabled May 19 15:00:47.502489 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 15:00:47.502489 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 15:00:47.546876 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: Python main interpreter initialized at 0x562125c6b9c0 May 19 15:00:47.546876 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: python threads support enabled May 19 15:00:47.546876 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: your server socket listen backlog is limited to 100 connections May 19 15:00:47.546876 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: your mercy for graceful operations on workers is 90 seconds May 19 15:00:47.547273 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: mapped 403077 bytes (393 KB) for 2 cores May 19 15:00:47.547273 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: *** Operational MODE: preforking *** May 19 15:00:47.547430 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: *** uWSGI is running in multiple interpreter mode *** May 19 15:00:47.547430 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: spawned uWSGI master process (pid: 277118) May 19 15:00:47.547656 dv-cinder-842542-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 15:00:47.549138 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: spawned uWSGI worker 1 (pid: 277130, cores: 1) May 19 15:00:47.549138 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: spawned uWSGI worker 2 (pid: 277131, cores: 1) May 19 15:00:47.549138 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 15:00:53.197814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 15:00:53.197814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: from cryptography.utils import int_from_bytes May 19 15:00:53.198777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 15:00:53.198777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: from cryptography.utils import int_from_bytes May 19 15:00:53.199252 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 15:00:53.199252 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: from cryptography.utils import int_from_bytes May 19 15:00:53.199399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 15:00:53.199399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: from cryptography.utils import int_from_bytes May 19 15:00:53.247534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 15:00:53.247765 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 15:00:53.247887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] command line args: [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 15:00:53.248201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 15:00:53.248384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 15:00:53.248488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 15:00:53.248602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ================================================================================ {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 15:00:53.248689 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] command line args: [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 15:00:53.248770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] admin_token = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.248850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 15:00:53.248938 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] config_dir = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.249018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ================================================================================ {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 15:00:53.249098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.249179 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] config_source = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.249309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] admin_token = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.249396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] debug = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.249484 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] config_dir = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.249564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] config_source = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250369 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] debug = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250488 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250605 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250749 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.250890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_config_append = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.251953 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_dir = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_config_append = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_file = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252252 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252589 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_dir = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252589 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_file = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252819 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.252958 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253901 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.253993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254502 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.254941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255654 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.255909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] publish_errors = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] publish_errors = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.256940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257142 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257252 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] standard_threads = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257509 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257694 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] standard_threads = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.257942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258050 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258050 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] use_journal = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258319 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] use_json = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] use_stderr = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258607 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] use_journal = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] use_syslog = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258807 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] use_json = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258807 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] use_stderr = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.258978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.259069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] use_syslog = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.259167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 15:00:53.259369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259455 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.259954 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260333 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260533 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260815 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260908 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.260994 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261301 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261683 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.261980 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262443 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262536 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.262947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263418 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263689 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.263947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264825 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.264921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265017 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265104 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265851 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.265981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266117 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266613 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266731 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.266974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.267086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.267202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.267690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.267808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.267941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268490 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.268983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.269889 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270007 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270125 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270496 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.270998 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271136 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271547 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271665 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.271913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.272988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.273105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.273346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.273475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.273594 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.273714 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.273855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274698 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.274986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275532 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.275955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.276102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.276241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.276877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.276985 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277225 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277495 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.277977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278093 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278225 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278472 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278569 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.278957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279056 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279152 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279491 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279684 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.279945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280314 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280585 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280879 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.280974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281060 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281256 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281645 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.281931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282017 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282562 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282655 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.282943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283320 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283490 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283596 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283689 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.external = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.283973 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.external = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284541 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.password = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284726 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.password = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] auth.token = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.284916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] auth.token = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285545 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285727 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285833 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.285927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286119 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286515 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286615 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.286965 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287056 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287537 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287819 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.287904 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.288994 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289098 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289466 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289794 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.289957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290101 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290491 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290593 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.290868 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291147 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291436 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291780 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.291987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292230 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292793 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.292943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293539 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.293909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294241 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294892 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.294993 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295482 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295660 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.295905 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296536 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.296873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297391 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297859 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.297986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298092 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298198 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298534 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298673 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.298953 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299236 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299474 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299840 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.299940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300321 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.300894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301502 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301609 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.301981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302080 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302553 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302802 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.302936 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303164 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303744 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303859 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.303965 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304185 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304436 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304631 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.304870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305524 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305762 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.305874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306329 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.306919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307603 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.307955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308109 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308657 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.308871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309008 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309506 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309607 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.309973 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310848 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.310981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311221 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311320 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311696 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311804 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.311929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312164 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312401 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312878 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.312980 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313383 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.313959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314087 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314187 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314563 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314672 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314795 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.314887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315603 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] role.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315836 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.315974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] role.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316073 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] role.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] role.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316781 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.316925 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.317927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318171 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318282 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318883 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.318989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.319126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.319236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.319368 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.319463 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.319613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.319783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.320996 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321228 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321594 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.321975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322204 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322528 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.322907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323016 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323157 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323866 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.323968 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324205 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324328 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324444 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324558 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.324918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325292 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325637 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326016 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326138 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326276 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.326989 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327451 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.327920 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328502 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.connection = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.328947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.connection = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329194 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329533 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.329910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330252 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330609 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.330938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331077 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331315 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.331876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332571 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.332954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.333049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.333144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 15:00:53.333259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 15:00:53.333349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 15:00:53.364554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.364792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.365015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.366662 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.366893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.367114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.374493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.375917 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.403166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.403407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.404242 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.404641 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.455274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.455653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.505148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.505477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.506317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.506530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.569464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.569675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.569928 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570559 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.570846 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.571082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.593193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.593497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.593671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.593771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.593881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.593966 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594053 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.594659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.605563 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.605776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.619869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620061 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620163 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620685 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.620951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.621033 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.621119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.621193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.621423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 15:00:53.638153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.638347 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 15:00:53.638757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.638931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 15:00:53.639040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.639213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 15:00:53.639370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.639470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.639568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.639662 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.639761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 15:00:53.639922 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.640064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.640233 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.640490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.640732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.651317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.651587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.651854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.652129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.652283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.652367 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.652674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.652946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.661088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.663287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.664388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.664671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.664955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.665437 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.665724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.665984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.666266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.666535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.666675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.666759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.667066 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.667223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.667335 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.667610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.668271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.668773 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.670650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.670650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.670650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.670650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.670650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.670650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.694200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.695815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.696096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.696827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.698544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.698831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.701423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.701709 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.701985 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.704290 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.704573 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.704840 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.709331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.709585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.712109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.712393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.714189 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.714472 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.714755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.715045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.715317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.715591 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.715861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.716124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.717017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.717330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.717599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.717861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.718143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.718403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.718674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.718929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.734767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.735047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.737489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.737761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.739562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.742258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.743192 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.743486 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.745880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.746167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.748487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.751142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.752071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.754718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.755847 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.756143 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.756429 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.756714 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.756992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.757294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.758450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.758745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.759021 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.759295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.759562 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.759933 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.776732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.777102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.777391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.777661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.779356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.779757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.780029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.780295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.785630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.787356 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.787630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.788171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.789898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.790172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.790721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.791001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.791283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.791587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.791858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.793186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.793490 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.793772 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.794046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.794327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.805274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.805578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.805858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.806134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.806411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.806695 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.806964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.807667 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.807953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.808236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.808512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.808832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.809039 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.809336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.827974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.830134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.831231 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.833413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.834544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.836132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.836740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.837788 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.838069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.838350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.838527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.840007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.840293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.840573 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.845710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.847322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.848026 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.849690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.850546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.850834 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.851118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.851389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.852924 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.853216 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.853523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.853800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.860682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.860943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.861268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.863175 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.863512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.863863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.870183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.870467 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.870739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.871009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.871276 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.871546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.871820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.871976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 15:00:53.872083 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.872173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.872344 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.872506 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.872615 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.872714 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.872880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873327 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873655 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.873818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.874096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.874344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.874616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.874883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.875154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.875417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 15:00:53.912624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562125c6b9c0 pid: 277131 (default app) May 19 15:00:53.914569 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562125c6b9c0 pid: 277130 (default app) May 19 15:00:53.916022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dd4e2-6015-4b1a-945b-9ba5ca56595a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:00:53.916316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dd4e2-6015-4b1a-945b-9ba5ca56595a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:00:53.916596 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dd4e2-6015-4b1a-945b-9ba5ca56595a None None] PATH_INFO: `/v3` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:00:53.920957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Thu May 19 15:00:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:01.981493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:01.982137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:01.982137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:01.986734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:01:01.993487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:01.994282 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:01.994454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:02.081082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: INFO dbcounter [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] Registered counter for database keystone May 19 15:01:02.106372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG oslo_db.sqlalchemy.engines [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277131) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 15:01:02.117714 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writer thread running {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 15:01:02.300564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG passlib.handlers.bcrypt [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277131) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 15:01:02.300987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG passlib.handlers.bcrypt [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277131) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 15:01:02.356836 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:02.358765 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:02.490077 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 15:01:02.513189 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:02.584543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: INFO dbcounter [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] Registered counter for database keystone May 19 15:01:02.607073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG oslo_db.sqlalchemy.engines [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277130) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 15:01:02.614718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writer thread running {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 15:01:02.736715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94933-3d65-409b-b105-2fb4ab15d31f None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:02.742169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97613977fc9648acbe5c5524c71c9426', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:02.742842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:02.743123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:02.743390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:02.752495 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:02.803417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 15:01:02.803731 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc475b-9584-4b32-bcee-4c1bac35e694 admin admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:02.814159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 15:01:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 15:01:02.820642 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:02.836403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09759f13-695d-4747-a8f6-86ca03fb7cc3 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:02.842019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97613977fc9648acbe5c5524c71c9426', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:02.843160 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:02.843443 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:02.843719 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] PATH_INFO: `/v3/projects/97613977fc9648acbe5c5524c71c9426` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:02.845434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=97613977fc9648acbe5c5524c71c9426)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:03.010776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 15:01:03.011515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3f4a3-ec2d-452e-90c8-7333882304f0 admin admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:03.015935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Thu May 19 15:01:02 2022] GET /identity/v3/projects/97613977fc9648acbe5c5524c71c9426 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:05.217818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c8b9f-1a35-4405-abd3-d1d27f750580 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:05.218543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c8b9f-1a35-4405-abd3-d1d27f750580 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:05.218543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c8b9f-1a35-4405-abd3-d1d27f750580 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:05.222641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:05.229783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:05.230485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:05.230678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:05.258159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:05.259709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:05.270632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 15:01:05.294239 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c8b9f-1a35-4405-abd3-d1d27f750580 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:05.314865 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c8b9f-1a35-4405-abd3-d1d27f750580 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:05.321904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee86e-3567-4f1b-9948-eafc031c3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97613977fc9648acbe5c5524c71c9426', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:05.322542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee86e-3567-4f1b-9948-eafc031c3a22 admin admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:05.322828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee86e-3567-4f1b-9948-eafc031c3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:05.323101 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee86e-3567-4f1b-9948-eafc031c3a22 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:05.332239 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee86e-3567-4f1b-9948-eafc031c3a22 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:05.334081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee86e-3567-4f1b-9948-eafc031c3a22 admin admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:05.343703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 19 15:01:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 15:01:05.350276 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:05.362372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96233a-53ab-43c9-9046-99ef47ef4a7f None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:05.368941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d687a56-68f4-4da2-88ed-551ce892289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97613977fc9648acbe5c5524c71c9426', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:05.369870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d687a56-68f4-4da2-88ed-551ce892289b admin admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:05.370156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d687a56-68f4-4da2-88ed-551ce892289b admin admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:05.370435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d687a56-68f4-4da2-88ed-551ce892289b admin admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:05.372091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d687a56-68f4-4da2-88ed-551ce892289b admin admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:05.374004 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d687a56-68f4-4da2-88ed-551ce892289b admin admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:05.375743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Thu May 19 15:01:05 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:07.697524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08c980-0466-4cc2-8058-4c2e4fd5ef9f None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:07.698129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08c980-0466-4cc2-8058-4c2e4fd5ef9f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:07.698129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08c980-0466-4cc2-8058-4c2e4fd5ef9f None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:07.699269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091eef4-e5cf-49a4-b1f6-363a233e5fc0 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:07.699691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:07.699776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091eef4-e5cf-49a4-b1f6-363a233e5fc0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:07.699859 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091eef4-e5cf-49a4-b1f6-363a233e5fc0 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:07.701211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:07.704750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:07.705011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:07.705319 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:07.706445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:07.706743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:07.707027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:07.724434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:07.725683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:07.745969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG passlib.handlers.bcrypt [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277130) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 15:01:07.746393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG passlib.handlers.bcrypt [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277130) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 15:01:07.792803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:07.794430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:07.806658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 7/12] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:07.830016 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:07.855040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:07.898470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 6/13] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:07.920794 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:07.938407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b160ef1-3de1-4554-9768-05d8f785612c None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:07.977067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:07.977849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:07.978134 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:07.978411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:07.991864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:08.003850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:08.005189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:08.006308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 8/14] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:08.012987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:08.104562 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:08.125138 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea3103-fae8-4f88-a11f-13fcf6cbf382 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.139863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070feac-aa80-47b0-b3bb-50b59c57151c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.142164 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:08.160130 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.160814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.161100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.161410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.162170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:08.162479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.163901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.176010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 9/15] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:08.178318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.179262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.179551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.179832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.181713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:08.249535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.250993 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.282191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484b4cf-aa17-40e5-858b-2a06682d6056 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.319532 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.320416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.320697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.320972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.322237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.323761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.325640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 10/16] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:08 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:08.331002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:08.345881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:08.360627 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 7/17] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:08.366179 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:08.368848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:08.433942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee40d76-0eb2-48d8-9ef9-7fbdb5549b6c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.465629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5ede5-12a1-499d-9b27-304bfdca2e1f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.466982 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.467793 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.468118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.468424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.469663 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.471147 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.472758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 11/18] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:08.478794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:08.498907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.499796 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.500043 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.500331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.520283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.521800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:01:08.542473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:08.543687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:08.544989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:01:08.544989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:01:08.544989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:01:08.544989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:08.544989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 8/19] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:08.551451 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:08.561680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:08.581231 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0817da5-dec4-416b-b6cd-a440410c3675 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.609851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:08.615297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.615970 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.616258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.616538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.617416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.domain.id=default, user.id=c7ff97c0da314994a8835ccf5732ab5e) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:08.617734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.619517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c2a81-0764-49ec-8811-058a1db08e50 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.626286 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 12/20] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 15:01:08 2022] GET /identity/v3/role_assignments?user.id=c7ff97c0da314994a8835ccf5732ab5e&scope.domain.id=default&role.id=8f859c3494964044956fd86bc077eaf0 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:08.655382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40927-8033-44c0-861c-ad22640651d6 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:08.694721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:08.695413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:08.695699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:08.695978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:08.696736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:08.697044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:08.698527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a60a22-d702-46cb-b68e-a68c5a60d2a6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:08.718934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 9/21] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:11.217847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9c856-dc0d-44dd-8e14-e8f75a7482d1 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.218480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9c856-dc0d-44dd-8e14-e8f75a7482d1 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.218694 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9c856-dc0d-44dd-8e14-e8f75a7482d1 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.220470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 13/22] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:11.226281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f7616-1fab-47ae-96e3-5af258761ccf None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.226860 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f7616-1fab-47ae-96e3-5af258761ccf None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.226860 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f7616-1fab-47ae-96e3-5af258761ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.250429 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-208f7616-1fab-47ae-96e3-5af258761ccf None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:11.251703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-208f7616-1fab-47ae-96e3-5af258761ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:11.354815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 10/23] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:11.390122 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9c856-dc0d-44dd-8e14-e8f75a7482d1 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:11.424235 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2a6b-5e37-4c2b-ab76-b8f307615a24 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.424788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2a6b-5e37-4c2b-ab76-b8f307615a24 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.425536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2a6b-5e37-4c2b-ab76-b8f307615a24 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.427808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 11/24] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:01:11.433340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.434081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.434534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.460642 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:11.462669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:11.563846 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:11.571120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9c856-dc0d-44dd-8e14-e8f75a7482d1 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:11.635008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:11.635995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.636285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.636567 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.643962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 12/25] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:11.655729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:11.663924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:11.665153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:11.666249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 14/26] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:11.674172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:11.682502 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:11.686657 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:11.817647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae9bec-68ae-429d-bc2f-de4dada0cb3c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:11.860970 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:11.861773 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.862065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.862399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.863195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:11.864674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:11.908306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8afbb-e6a0-4d7c-9ddf-bec4a01748e1 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:11.978896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:11.979651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:11.979937 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:11.980219 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:11.981002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:11.981338 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:11.982758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ee4c4-ec6f-453b-9a7f-a3b54b5339f6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.004335 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 13/27] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:12.008201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 15/28] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 15:01:11 2022] POST /identity/v3/roles => generated 219 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:12.010298 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6e98-e550-4001-9181-bcd27fe83635 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.010862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6e98-e550-4001-9181-bcd27fe83635 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.011368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6e98-e550-4001-9181-bcd27fe83635 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.013308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 14/29] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:12.014607 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.025208 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d3b1-98f3-4ee9-a82e-452ec3696422 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.025894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d3b1-98f3-4ee9-a82e-452ec3696422 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.026421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d3b1-98f3-4ee9-a82e-452ec3696422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.074611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-bab7d3b1-98f3-4ee9-a82e-452ec3696422 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:12.076270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-bab7d3b1-98f3-4ee9-a82e-452ec3696422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:12.095858 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:12.137587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:12.150531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd99f3-3836-4d40-ba8f-da436cc95625 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:12.225977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bca8-8112-4e4a-bb75-9e6ca0677cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:12.227759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bca8-8112-4e4a-bb75-9e6ca0677cd7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.228278 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bca8-8112-4e4a-bb75-9e6ca0677cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.228796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bca8-8112-4e4a-bb75-9e6ca0677cd7 None admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.231785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6bca8-8112-4e4a-bb75-9e6ca0677cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:12.233950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6bca8-8112-4e4a-bb75-9e6ca0677cd7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.238866 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 16/30] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:12 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:12.245359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358351a-7f07-4586-afb0-1fcea0bf31f7 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.245359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358351a-7f07-4586-afb0-1fcea0bf31f7 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.245359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358351a-7f07-4586-afb0-1fcea0bf31f7 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.245359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 17/31] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:12.255069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358351a-7f07-4586-afb0-1fcea0bf31f7 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.256221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 15/32] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:12.259290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.261969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.262491 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.292923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:12.295010 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:12.396984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 16/33] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:12.398162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.417728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358351a-7f07-4586-afb0-1fcea0bf31f7 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:12.463117 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:12.472848 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:12.477006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.477881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.478470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.480187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:12.483151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.485215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 18/34] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:12.486355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.543208 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:12.549227 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5b5a4-545b-4921-99a6-5069e3dfc1c1 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:12.594267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:12.595400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.595693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.595972 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.598849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90192cd5-f7d1-4332-b966-5e8c5057ac39 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:12.609142 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:12.610988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:01:12.634120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:12.635223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:12.636297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:01:12.636297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:01:12.636297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:01:12.636297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:12.636297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 17/35] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:12.636297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.643695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544130ea-e63a-46a4-9af2-d9bdd95cc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:12.644417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544130ea-e63a-46a4-9af2-d9bdd95cc0c8 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.644707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544130ea-e63a-46a4-9af2-d9bdd95cc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.644979 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544130ea-e63a-46a4-9af2-d9bdd95cc0c8 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.645887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544130ea-e63a-46a4-9af2-d9bdd95cc0c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:12.647399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544130ea-e63a-46a4-9af2-d9bdd95cc0c8 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.662969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 19/36] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 15:01:12 2022] POST /identity/v3/roles => generated 225 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:12.664385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa634cb-037c-4d12-b219-223cea3ae679 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.664675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa634cb-037c-4d12-b219-223cea3ae679 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.664954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa634cb-037c-4d12-b219-223cea3ae679 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.666452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 20/37] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:12.671591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa634cb-037c-4d12-b219-223cea3ae679 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.764496 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:12.779256 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59039e35-2004-4025-9430-984c5fb41810 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:12.821104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:12.821975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.822261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.822609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] PATH_INFO: `/v3/domains/default/users/c7ff97c0da314994a8835ccf5732ab5e/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.825495 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c7ff97c0da314994a8835ccf5732ab5e, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:12.827168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.829626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa634cb-037c-4d12-b219-223cea3ae679 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:12.870699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:12.871785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.872071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.872362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.873120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:12.874266 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:12.875700 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2e356-5952-44cd-9aba-a99ba4aadc5a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:12.898554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 21/38] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:12.900294 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:12.900832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:12.901386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:12.923909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:12.927717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:12.971203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 18/39] 10.222.0.73 () {58 vars in 1361 bytes} [Thu May 19 15:01:12 2022] PUT /identity/v3/domains/default/users/c7ff97c0da314994a8835ccf5732ab5e/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:12.972468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:12.991777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:13.013468 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 22/40] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:13.043668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:13.098746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:13.112896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ba7f4-d04c-479c-9efc-1db473a4d968 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:13.151406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead8d3a-a071-4767-9b02-98d1c6e2b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:13.151990 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8d3a-a071-4767-9b02-98d1c6e2b314 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:13.152286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8d3a-a071-4767-9b02-98d1c6e2b314 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:13.152561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8d3a-a071-4767-9b02-98d1c6e2b314 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:13.153365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead8d3a-a071-4767-9b02-98d1c6e2b314 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:13.155100 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead8d3a-a071-4767-9b02-98d1c6e2b314 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:13.155934 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ea4a-00a6-4eac-8abc-ec61dfeed6b4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:13.196419 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 19/41] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:01:12 2022] POST /identity/v3/projects => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:13.196985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:13.197710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:13.197891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd543bb-1818-4160-8b63-3cbb91f95931 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:13.197998 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:13.198435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd543bb-1818-4160-8b63-3cbb91f95931 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:13.198606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:13.198687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd543bb-1818-4160-8b63-3cbb91f95931 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:13.199143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:13.200082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 20/42] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:13.204827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:13.206864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:13.207150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:13.207443 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:13.226973 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 23/43] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 15:01:13 2022] POST /identity/v3/roles => generated 223 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 15:01:13.233593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:13.235760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:13.312339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:13.333036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 21/44] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:13.352077 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:13.473266 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8c8a-97c1-4c71-97b9-ff08bb00977b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:13.486353 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:13.513328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6fbe-8edd-4a23-8b22-2e5d07ed86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:13.514168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6fbe-8edd-4a23-8b22-2e5d07ed86ce None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:13.514458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6fbe-8edd-4a23-8b22-2e5d07ed86ce None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:13.514763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6fbe-8edd-4a23-8b22-2e5d07ed86ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:13.516020 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6fbe-8edd-4a23-8b22-2e5d07ed86ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:13.517914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6fbe-8edd-4a23-8b22-2e5d07ed86ce None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:13.519752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 24/45] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:13.525427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:13.630753 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03e9c3-bc55-457c-8c12-888c1c088678 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:13.669141 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e6cdd-065b-40f3-91f2-7e61e74ddea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:13.669849 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e6cdd-065b-40f3-91f2-7e61e74ddea0 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:13.670137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e6cdd-065b-40f3-91f2-7e61e74ddea0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:13.670427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e6cdd-065b-40f3-91f2-7e61e74ddea0 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:13.671655 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e6cdd-065b-40f3-91f2-7e61e74ddea0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:13.673269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e6cdd-065b-40f3-91f2-7e61e74ddea0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:13.700272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 22/46] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:01:13 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 15:01:15.224398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a655-2f28-4c00-b804-0db4286bfc5d None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.224999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a655-2f28-4c00-b804-0db4286bfc5d None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.224999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a655-2f28-4c00-b804-0db4286bfc5d None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.226401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 25/47] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:15.231932 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.232380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.232500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.250156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:15.251364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:15.297775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:15.326272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 23/48] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:15.349375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22a655-2f28-4c00-b804-0db4286bfc5d None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:15.459619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22a655-2f28-4c00-b804-0db4286bfc5d None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:15.495373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:15.496106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.496399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.496681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.507620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:15.515361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:15.516416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:15.518158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 26/49] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:15.522962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:15.627226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd59f3-f35c-4005-b143-5dbd15c53771 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:15.664851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:15.665576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.665844 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.666153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.666917 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:15.667202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:15.668618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:15.677081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 24/50] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:15.682927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:15.725272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:15.790939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87673-dddc-448e-a50b-095a1a1183a8 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:15.826463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb1ea5-e32a-41ac-84ab-1422de6989a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:15.827460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb1ea5-e32a-41ac-84ab-1422de6989a8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.827712 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb1ea5-e32a-41ac-84ab-1422de6989a8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.828002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb1ea5-e32a-41ac-84ab-1422de6989a8 None admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.829258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb1ea5-e32a-41ac-84ab-1422de6989a8 None admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:15.830747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb1ea5-e32a-41ac-84ab-1422de6989a8 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:15.832446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 27/51] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:15 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:15.838359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:15.875638 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:15.915945 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba091a1-53ce-46c5-86d5-11b457cd19e3 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.916201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba091a1-53ce-46c5-86d5-11b457cd19e3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.916508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba091a1-53ce-46c5-86d5-11b457cd19e3 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.917940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 28/52] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:15.923194 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.923493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.923731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.941523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:15.942747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:15.944042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfd38a-d845-43b3-8ccc-c29ed63db230 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:15.983047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:15.984348 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:15.984628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:15.984914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:15.989071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:15.989071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:15.989432 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 25/53] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:15.995479 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:16.025514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 29/54] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:16.049074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:16.084142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:16.103258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9e69-0ea3-410a-852d-dca6b00690b1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:16.106039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:16.140429 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:16.141101 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:16.141433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:16.141715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:16.142935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.domain.id=default, user.id=c7ff97c0da314994a8835ccf5732ab5e) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:16.143255 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:16.144735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:16.153104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 26/55] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 19 15:01:15 2022] GET /identity/v3/role_assignments?user.id=c7ff97c0da314994a8835ccf5732ab5e&scope.domain.id=default&role.id=8f859c3494964044956fd86bc077eaf0 => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:16.161643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00043bc-05a6-46dc-94aa-5ffdfdaf3068 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:16.203208 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932c565-9110-4ffa-b8c9-e4534339d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:16.204061 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c565-9110-4ffa-b8c9-e4534339d61f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:16.204351 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c565-9110-4ffa-b8c9-e4534339d61f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:16.204637 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c565-9110-4ffa-b8c9-e4534339d61f None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:16.206018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932c565-9110-4ffa-b8c9-e4534339d61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:16.207664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932c565-9110-4ffa-b8c9-e4534339d61f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:16.210442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 30/56] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:16.215393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:16.335904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb475dc-cbea-4f40-a4c6-d1c3269b6fbd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:16.371577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2dc93b-4e47-4aff-b104-1a7e2f05ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:16.372248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dc93b-4e47-4aff-b104-1a7e2f05ab33 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:16.372539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dc93b-4e47-4aff-b104-1a7e2f05ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:16.372818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dc93b-4e47-4aff-b104-1a7e2f05ab33 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:16.373606 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2dc93b-4e47-4aff-b104-1a7e2f05ab33 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:16.375185 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2dc93b-4e47-4aff-b104-1a7e2f05ab33 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:16.403678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 27/57] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 15:01:16 2022] POST /identity/v3/projects => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:18.618680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74abf-924d-4c97-b6ba-447cc0e6de4a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:18.619272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74abf-924d-4c97-b6ba-447cc0e6de4a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:18.619272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74abf-924d-4c97-b6ba-447cc0e6de4a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:18.620676 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 31/58] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:18.627507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:18.628258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:18.628426 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:18.657170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:18.658432 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:18.673369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:18.736347 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 28/59] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:18.761065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d74abf-924d-4c97-b6ba-447cc0e6de4a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:18.872556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d74abf-924d-4c97-b6ba-447cc0e6de4a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:18.874896 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:18.908582 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836a60f-84d7-43d7-a6e3-a8bde6b6a3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:18.909417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836a60f-84d7-43d7-a6e3-a8bde6b6a3ca None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:18.909703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836a60f-84d7-43d7-a6e3-a8bde6b6a3ca None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:18.909981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836a60f-84d7-43d7-a6e3-a8bde6b6a3ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:18.911234 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836a60f-84d7-43d7-a6e3-a8bde6b6a3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:18.912735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836a60f-84d7-43d7-a6e3-a8bde6b6a3ca None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:18.914400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 32/60] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:18.920240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:19.032114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93654b9b-4919-4a06-b13d-79a673d59474 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:19.069572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea52356-b46a-4b89-8768-6d4dcf33cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:19.070254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea52356-b46a-4b89-8768-6d4dcf33cdae None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:19.070546 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea52356-b46a-4b89-8768-6d4dcf33cdae None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:19.070831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea52356-b46a-4b89-8768-6d4dcf33cdae None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:19.071574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea52356-b46a-4b89-8768-6d4dcf33cdae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:19.073070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea52356-b46a-4b89-8768-6d4dcf33cdae None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:19.100691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 29/61] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:18 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:22.278315 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0156840-b0ab-4b4f-a017-3c31b8272de8 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:22.279214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0156840-b0ab-4b4f-a017-3c31b8272de8 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:22.279742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0156840-b0ab-4b4f-a017-3c31b8272de8 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:22.281412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 33/62] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:22.287642 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40a60-11a4-4cae-bcab-4035b1f428c6 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:22.288098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40a60-11a4-4cae-bcab-4035b1f428c6 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:22.288290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40a60-11a4-4cae-bcab-4035b1f428c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:22.306619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:22.314364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-1ee40a60-11a4-4cae-bcab-4035b1f428c6 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:22.315595 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-1ee40a60-11a4-4cae-bcab-4035b1f428c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:22.407819 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 30/63] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:22.428971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda759c-ea86-44e0-93b8-5b8cf69da087 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:22.429666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda759c-ea86-44e0-93b8-5b8cf69da087 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:22.430165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda759c-ea86-44e0-93b8-5b8cf69da087 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:22.431811 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 34/64] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:22.437031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad769360-0303-42be-91cd-421236b81f76 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:22.437350 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad769360-0303-42be-91cd-421236b81f76 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:22.437629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad769360-0303-42be-91cd-421236b81f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:22.445189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda759c-ea86-44e0-93b8-5b8cf69da087 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:22.464667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-ad769360-0303-42be-91cd-421236b81f76 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:22.467440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-ad769360-0303-42be-91cd-421236b81f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:22.601857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 31/65] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:22.625684 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda759c-ea86-44e0-93b8-5b8cf69da087 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:22.636551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad769360-0303-42be-91cd-421236b81f76 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:22.709622 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:22.710875 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:22.711390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:22.711888 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:22.733510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:22.749829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:22.750995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:22.751960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:22.761556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 35/66] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:22.767851 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:22.773097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:22.820214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:22.835535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad769360-0303-42be-91cd-421236b81f76 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:22.887973 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:22.889321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:22.890863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:22.891458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:22.917913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:22.928440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:22.929610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:22.930559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:22.939019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 32/67] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:22.946584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:22.959416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e877d58-250c-4912-8580-4ebc2e1bb358 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.039969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.041063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.041624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.042629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.044154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:23.044695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.047154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a0750-298c-4847-822d-6122f8fd843c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.057207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 36/68] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:23.058951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a3b64-9aee-41d2-8cd8-2adcf0a1e6ed None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.059569 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a3b64-9aee-41d2-8cd8-2adcf0a1e6ed None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.060069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a3b64-9aee-41d2-8cd8-2adcf0a1e6ed None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.061595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 37/69] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:23.063133 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a3b64-9aee-41d2-8cd8-2adcf0a1e6ed None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.116638 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726866de-ba1a-49f0-87c0-688f949f5c07 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.144916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.154073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.155134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.155434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.155725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.156507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:23.156826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.158509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30615b-c06d-40df-89d9-0c650ef7adb2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.167057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 33/70] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:23.167247 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.168344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.168559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.168977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.176849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a3b64-9aee-41d2-8cd8-2adcf0a1e6ed None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.186138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:23.187408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:23.211045 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207405a-cbd5-4818-b229-c3f88e59b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.211941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207405a-cbd5-4818-b229-c3f88e59b57e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.212232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207405a-cbd5-4818-b229-c3f88e59b57e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.212513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207405a-cbd5-4818-b229-c3f88e59b57e None admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.213786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207405a-cbd5-4818-b229-c3f88e59b57e None admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.215261 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207405a-cbd5-4818-b229-c3f88e59b57e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.217079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 38/71] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:23.218249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfec5-f47f-4a68-a9bf-96b625d915cc None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.218578 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfec5-f47f-4a68-a9bf-96b625d915cc None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.218821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfec5-f47f-4a68-a9bf-96b625d915cc None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.220060 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 39/72] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:23.221125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb4d4c-7ce1-4dc4-b020-248d2d1b5de6 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.221972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb4d4c-7ce1-4dc4-b020-248d2d1b5de6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.221972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb4d4c-7ce1-4dc4-b020-248d2d1b5de6 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.223237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 40/73] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:23.223986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb4d4c-7ce1-4dc4-b020-248d2d1b5de6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.258975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 34/74] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:23.259829 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.347999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb4d4c-7ce1-4dc4-b020-248d2d1b5de6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.371360 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82672c-33a3-4e1d-97c1-b1656eaffd53 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.383899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2902e-1348-4075-bc21-3be98f89a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.386897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2902e-1348-4075-bc21-3be98f89a5a0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.386897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2902e-1348-4075-bc21-3be98f89a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.386897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2902e-1348-4075-bc21-3be98f89a5a0 None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.393287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.401787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2902e-1348-4075-bc21-3be98f89a5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.403214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2902e-1348-4075-bc21-3be98f89a5a0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.405154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 41/75] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:23.406202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eaff7-924c-461a-8893-b5a940fef6d0 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.406482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eaff7-924c-461a-8893-b5a940fef6d0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.406773 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eaff7-924c-461a-8893-b5a940fef6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.415795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.416457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.416748 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.417019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.422624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-b77eaff7-924c-461a-8893-b5a940fef6d0 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:23.423887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-b77eaff7-924c-461a-8893-b5a940fef6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:23.438104 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.489134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.490741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.491752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 42/76] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:23.494664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.495171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.495271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.512992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:23.514243 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:23.582743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 43/77] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:23.584248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.592930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 35/78] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:23.596178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.615468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.665537 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.684654 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45641a-79f2-4fb0-b0cd-971a21926393 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.697681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef9b5-a528-4d37-a3a9-1b826d31c357 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.717902 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.718740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.719032 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.719316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.733555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.733793 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:23.734217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.734508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.734794 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:23.742311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:23.743386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:23.744362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 44/79] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:23.745228 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.805214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.806736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.825313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.847665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87321a-9826-46d1-8fd5-52dcf8bbb6b8 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:23.881178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:23.882006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:23.882294 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:23.882565 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:23.883791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:23.885293 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:23.886947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 45/80] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:23.890266 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.901958 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:23.907864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 36/81] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:23.908730 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:23.992982 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20422336-24d1-4b9d-ace9-239f0a198bbf None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.008237 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f93ab-7668-444a-b6b0-ca89ff07cd3e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.027882 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.033022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.033943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.034236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.034519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.035766 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.037279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.039016 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 46/82] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.039781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.049788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.050971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.051266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.051552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.062211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:24.070068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:24.071102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 37/83] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:24.072260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.121470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.145443 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de6c60-5e9c-4683-ac3d-82a38816c9ec None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.169847 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972032de-8661-4ccc-a14a-9a1f58f445f1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.178286 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.178971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.179263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.179550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.180320 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:24.180643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.182101 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.190306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 47/84] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.191914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.205199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.205921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.206202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.206488 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.207218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.208734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.228668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 38/85] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:24 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:24.229693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.275932 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.306516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a95e97-9897-4c29-b108-115cb3b74036 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.344346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.345356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.345648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.345931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.347322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.348847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.350685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 48/86] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.351489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.361084 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.361460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c073dd-f992-4160-99b4-af375c6aa0fc None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.405086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.405858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.406122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.406375 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.407310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=c7ff97c0da314994a8835ccf5732ab5e) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:24.407626 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.409127 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.415291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 39/87] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/role_assignments?user.id=c7ff97c0da314994a8835ccf5732ab5e&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.423117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.479764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a741c-b9b0-4c32-b461-46dc54b47035 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.524876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.526064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.526597 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.527110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.528161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] RBAC: Adding query filter params (name=member) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:24.528716 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.531079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.545581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 49/88] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.545581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.557387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.563843 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e0345-57e6-4766-9f3c-f46b6ca0d150 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.607958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.609355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.609826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.610342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.611866 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.613690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.615722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 40/89] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.616725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.643103 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.687309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1a9dd-4c54-4ed1-bc81-1b1ffbcc4ba1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.725619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650462e-4d72-477e-b160-2183286232db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.726308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650462e-4d72-477e-b160-2183286232db None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.726592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650462e-4d72-477e-b160-2183286232db None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.726891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650462e-4d72-477e-b160-2183286232db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.727733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650462e-4d72-477e-b160-2183286232db None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=96c2186a1d27432e8e831c999d91f365) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:24.728057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650462e-4d72-477e-b160-2183286232db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.729608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650462e-4d72-477e-b160-2183286232db None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.735841 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 50/90] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/role_assignments?user.id=96c2186a1d27432e8e831c999d91f365&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.736547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c498ee-cf4b-4e83-b2b6-bc667b3bc0ee None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.736687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650462e-4d72-477e-b160-2183286232db None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.776371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.777665 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.778168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.778602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:24.779979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:24.781780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:24.783777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 41/91] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:24.789376 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:24.827209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.921428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650462e-4d72-477e-b160-2183286232db None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.963088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2915452-616b-4b2a-8f8a-2706e74a7af1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:24.984111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:24.992925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:24.996102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:24.997500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:24.998109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:25.035476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:25.036598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:25.037900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:25.038482 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:25.131589 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:25.134411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:25.182084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:25.183870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:25.264339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:25.317327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 51/92] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:25.322631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:25.389603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:25.461557 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 42/93] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 15:01:25.468341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:25.563308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f0e29-10d1-4821-8e3f-9433293526ea None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:25.652335 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:25.653254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:25.653604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:25.653896 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:25.655261 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:25.656817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:25.658632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 52/94] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:25 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:25.665137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:25.724587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96e9c-5cda-4925-9939-29d371f2ebf1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:25.779536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:25.780436 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:25.780727 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:25.781009 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] PATH_INFO: `/v3/projects/8b0c45536f43454eb65129d052a30ffd` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:25.782512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] RBAC: Authorizing `identity:get_project(project_id=8b0c45536f43454eb65129d052a30ffd)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:25.784156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:25.786119 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 43/95] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:25 2022] GET /identity/v3/projects/8b0c45536f43454eb65129d052a30ffd => generated 312 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:25.793469 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:25.801258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:25.893686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:25.897442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47f78-81c1-460f-983b-40f932b2027e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:25.963735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:25.964811 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:25.965365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:25.965898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:25.967001 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] RBAC: Adding query filter params (role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=96c2186a1d27432e8e831c999d91f365) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:25.967527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:25.969267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1470d6-0695-4cff-9ff1-7f01af17ec1a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:25.976089 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 53/96] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:25 2022] GET /identity/v3/role_assignments?user.id=96c2186a1d27432e8e831c999d91f365&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:26.019756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8d702-438b-4095-9651-e2ba526ea36b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:26.071153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:26.071838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:26.072125 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:26.072404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:26.073296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=8b0c45536f43454eb65129d052a30ffd, user.id=96c2186a1d27432e8e831c999d91f365) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:26.073619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:26.075122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e199d4-ee64-42b1-8111-a47b5f206aca None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:26.086158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 44/97] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:25 2022] GET /identity/v3/role_assignments?user.id=96c2186a1d27432e8e831c999d91f365&scope.project.id=8b0c45536f43454eb65129d052a30ffd&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:27.390740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2538c-26a1-45ff-9eea-80e9b016f697 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:27.391871 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2538c-26a1-45ff-9eea-80e9b016f697 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:27.392734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2538c-26a1-45ff-9eea-80e9b016f697 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:27.394709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 54/98] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:27.402455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:27.402979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:27.403073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:27.426242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:27.427470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:27.525729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:27.535680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 45/99] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:27.559114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2538c-26a1-45ff-9eea-80e9b016f697 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:27.685025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2538c-26a1-45ff-9eea-80e9b016f697 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:27.738603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba535f5-4638-4c97-afb4-13703c48259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:27.739445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba535f5-4638-4c97-afb4-13703c48259f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:27.739739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba535f5-4638-4c97-afb4-13703c48259f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:27.740023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba535f5-4638-4c97-afb4-13703c48259f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:27.756483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba535f5-4638-4c97-afb4-13703c48259f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7ba535f5-4638-4c97-afb4-13703c48259f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:27.764450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:27.765648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:27.766800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 55/100] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:27.774520 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:27.821528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140879cf-e671-4b3f-8194-4e0098d05d68 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:27.821744 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140879cf-e671-4b3f-8194-4e0098d05d68 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:27.822028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140879cf-e671-4b3f-8194-4e0098d05d68 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:27.823429 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 56/101] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:27.829027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c020-5887-49c1-b84a-738286ef4d0b None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:27.829361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c020-5887-49c1-b84a-738286ef4d0b None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:27.829655 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c020-5887-49c1-b84a-738286ef4d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:27.854855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-2e55c020-5887-49c1-b84a-738286ef4d0b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:27.856090 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-2e55c020-5887-49c1-b84a-738286ef4d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:27.898129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:27.931570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eddfb-af8b-4404-8284-ae16fa790bab None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:27.963579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 57/102] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:27.971540 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5d8c-9961-445e-8b7b-a878f3df081b None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:27.971838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5d8c-9961-445e-8b7b-a878f3df081b None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:27.972127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5d8c-9961-445e-8b7b-a878f3df081b None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:27.973481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 58/103] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:28.001623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.002318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.002605 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.002890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.003663 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] RBAC: Adding query filter params (name=reader) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:28.003976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:28.005442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d8460-8f07-47fb-9142-b124a2c427df None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:28.012075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.012377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.012664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.037281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 46/104] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:28.038844 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9df0-eac8-4a30-a1eb-0828d7a835fe None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.039145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9df0-eac8-4a30-a1eb-0828d7a835fe None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.039432 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9df0-eac8-4a30-a1eb-0828d7a835fe None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.040401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:28.040691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 47/105] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:28.041478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef9df0-eac8-4a30-a1eb-0828d7a835fe None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.045281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:28.112485 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:28.120831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 59/106] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:28.126172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.161337 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef9df0-eac8-4a30-a1eb-0828d7a835fe None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.214031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb0cb9-0e75-4544-ad4d-f4134bd20fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.214855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb0cb9-0e75-4544-ad4d-f4134bd20fff None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.215147 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb0cb9-0e75-4544-ad4d-f4134bd20fff None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.215434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb0cb9-0e75-4544-ad4d-f4134bd20fff None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.230451 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb0cb9-0e75-4544-ad4d-f4134bd20fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:28.237806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-36eb0cb9-0e75-4544-ad4d-f4134bd20fff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:28.241921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:28.242959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:28.243910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 48/107] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:28.244694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.245203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.245203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.260029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faa7fb-5e94-4ba1-a7f3-e5225c2e1fa4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.269568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:28.270799 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:28.299008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.300291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.300799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.301324 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] PATH_INFO: `/v3/users/d7710a2b5d1c4650a14a9c8cb5d43b71` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.316676 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] RBAC: Authorizing `identity:get_user(user_id=d7710a2b5d1c4650a14a9c8cb5d43b71)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:28.318542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:28.320710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 60/108] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/users/d7710a2b5d1c4650a14a9c8cb5d43b71 => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:28.325409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.363769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 49/109] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:28.365438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.429598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:28.465053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36fecd-576f-4526-9d52-f6b84637b3cf None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.486558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf5b5c-51d6-4ffc-a6f1-43a0163bfb11 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.510040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.511629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.512522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.512909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.514901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:28.517714 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:28.520026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 61/110] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:28.521571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.526158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.527135 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.527658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.528174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.529172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:28.529742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:28.531438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:28.537731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:28.542519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 50/111] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:28.549868 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.628687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d41fe1-0368-4a60-afdf-07ef87a964fc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.661669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36564e22-6fd5-41f8-845a-97b5005217b3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.667721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.668413 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.668705 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.668990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.676779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:28.711482 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.712714 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.713625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.713917 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.728955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:28.740616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:28.741798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:28.742733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 51/112] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:28.743532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.751012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:28.752918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:28.765411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:28.864847 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b4b07-aaaf-4c98-9538-c8289c1734ed None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:28.865964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 62/113] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:28.870204 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:28.901136 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:28.901769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:28.902056 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:28.902349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:28.903076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:28.904557 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:28.932016 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 52/114] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:01:28 2022] POST /identity/v3/projects => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:28.932724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.001640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.006554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2428a94-7985-4954-ba25-cd89a803da9f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.026475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.054007 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24820dd1-b1fb-4d34-bafb-c2c09b4ec270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.055465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24820dd1-b1fb-4d34-bafb-c2c09b4ec270 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.055968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24820dd1-b1fb-4d34-bafb-c2c09b4ec270 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.056455 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24820dd1-b1fb-4d34-bafb-c2c09b4ec270 None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.058169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24820dd1-b1fb-4d34-bafb-c2c09b4ec270 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.059877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24820dd1-b1fb-4d34-bafb-c2c09b4ec270 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.062029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 63/115] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.064154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0187b8a-cb50-4933-a40f-4a3dffa70770 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.064679 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0187b8a-cb50-4933-a40f-4a3dffa70770 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.065159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0187b8a-cb50-4933-a40f-4a3dffa70770 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.067100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 64/116] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:29.068112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0187b8a-cb50-4933-a40f-4a3dffa70770 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.087270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343498a4-3d5f-440a-8e39-4c1c76fc8316 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.139674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.140475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.140830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.141350 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.142538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:29.143688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.145135 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.154896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 53/117] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.155803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.204834 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0187b8a-cb50-4933-a40f-4a3dffa70770 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.250891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c660ed-f108-4254-922e-7c2cc34668f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.252274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c660ed-f108-4254-922e-7c2cc34668f8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.252566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c660ed-f108-4254-922e-7c2cc34668f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.253127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c660ed-f108-4254-922e-7c2cc34668f8 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.254737 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.255510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c660ed-f108-4254-922e-7c2cc34668f8 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.257476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c660ed-f108-4254-922e-7c2cc34668f8 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.259595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 65/118] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.261278 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.261579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.261862 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.286755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a3560-2e06-41f9-a301-e757f7cb62c4 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.287354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:29.288939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:29.324013 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.325005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.325400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.325749 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.327024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.328589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.330686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 54/119] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.331276 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.342038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.370367 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 66/120] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:29.371456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.459693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dbdda-2daa-4b7e-89e7-abc775cb51af None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.476682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79ac62-721f-484e-a8cb-c10f95fd82ea None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.495150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104b18f-d1b8-41b1-a233-d4001ea9b966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.496046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b18f-d1b8-41b1-a233-d4001ea9b966 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.496336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b18f-d1b8-41b1-a233-d4001ea9b966 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.496628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b18f-d1b8-41b1-a233-d4001ea9b966 None admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.497906 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b18f-d1b8-41b1-a233-d4001ea9b966 None admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.499400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b18f-d1b8-41b1-a233-d4001ea9b966 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.501403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 55/121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.506899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7987e-70aa-48d8-8d93-60d81eaa104d None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.507211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7987e-70aa-48d8-8d93-60d81eaa104d None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.507483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7987e-70aa-48d8-8d93-60d81eaa104d None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.508990 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 56/122] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:29.512268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.513039 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.513285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.513538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7987e-70aa-48d8-8d93-60d81eaa104d None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.513670 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.514410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] RBAC: Adding query filter params (role.id=f89d9bb93fd142f4bb72e322d8502a85, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=d7710a2b5d1c4650a14a9c8cb5d43b71) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:29.514729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.516314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.522905 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 67/123] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/role_assignments?user.id=d7710a2b5d1c4650a14a9c8cb5d43b71&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=f89d9bb93fd142f4bb72e322d8502a85 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.523545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.550007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.595218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.652271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7987e-70aa-48d8-8d93-60d81eaa104d None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.670113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11a50c-1eb4-4df2-bd37-a7b311188434 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.689974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.691018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.691313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.691595 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/96c2186a1d27432e8e831c999d91f365/roles/6cb27cd3219a4df396848116d7e9df7e` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.693842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac748ce4db4449de9948ac8e1f93cc8b, user_id=96c2186a1d27432e8e831c999d91f365, role_id=6cb27cd3219a4df396848116d7e9df7e)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.695509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.712023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd182ccf-4a1e-4922-8661-af4ab6f2c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.712250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 57/124] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:29 2022] PUT /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/96c2186a1d27432e8e831c999d91f365/roles/6cb27cd3219a4df396848116d7e9df7e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:29.712959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.713097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182ccf-4a1e-4922-8661-af4ab6f2c1de None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.713600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182ccf-4a1e-4922-8661-af4ab6f2c1de None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.714229 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182ccf-4a1e-4922-8661-af4ab6f2c1de None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.729080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd182ccf-4a1e-4922-8661-af4ab6f2c1de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-fd182ccf-4a1e-4922-8661-af4ab6f2c1de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:29.738825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:29.740149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:29.741313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:29.749262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 68/125] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:29.749262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.749262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.749262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.766192 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:29.768227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:29.825217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.851558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87104335-9bfa-4593-b2ca-3525f398a340 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:29.855113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 69/126] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:29.856276 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.890616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:29.895082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:29.895994 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:29.896534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:29.896744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:29.898922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:29.900434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:29.902327 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 58/127] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:29.903146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:29.977937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc73574-0ec0-4eba-8eeb-8951a16916d0 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.020937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.021613 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.021905 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.022187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.023002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] RBAC: Adding query filter params (name=member) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:30.023277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.024703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.034257 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 70/128] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:30.034864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.070837 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b6c1b-f4d9-4cfc-a859-96ca23815c88 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.108126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.109407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.109951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.110493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.122746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:30.131483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:30.132531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:30.133891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:30.134616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:30.136549 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 59/129] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:01:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:30.137617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.144051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74441110-5478-41c2-a471-c69da7d41a41 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.155519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:30.180686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.181720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.182013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.182302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/c7ff97c0da314994a8835ccf5732ab5e/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.184470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] RBAC: Authorizing `identity:create_grant(project_id=ac748ce4db4449de9948ac8e1f93cc8b, user_id=c7ff97c0da314994a8835ccf5732ab5e, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.186819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.225012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 71/130] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:30 2022] PUT /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/c7ff97c0da314994a8835ccf5732ab5e/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:30.225755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.288118 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb75466-a339-4a25-8ac3-9573876e1a6c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.382112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.383042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.383331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.383615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.384950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.386457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.388167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 60/131] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:30 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:30.394087 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.480206 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3f83d-af64-4eba-80ec-ae9ff3b600fe None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.563802 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:30.586478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.587656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.588172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.589043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.590228 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:30.591975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.593688 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.608651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 72/132] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:01:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:30.609991 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0c48-a25d-4334-9c4a-eb653c938f6e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.615162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.661627 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:30.666248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.667442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.668144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.669977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] PATH_INFO: `/v3/projects/8b0c45536f43454eb65129d052a30ffd` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.672219 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] RBAC: Authorizing `identity:get_project(project_id=8b0c45536f43454eb65129d052a30ffd)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.676228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.681971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 61/133] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:30 2022] GET /identity/v3/projects/8b0c45536f43454eb65129d052a30ffd => generated 312 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:30.687074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.831388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7525c-7e82-4293-8dac-8e05596621a1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.884159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08375-a53f-4a85-8fc8-66d63a95ec4f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:30.902503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.903853 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.904424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.904920 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.907449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.914964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.918067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 73/134] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:30 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:30.924385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:30.964717 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:30.966121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:30.966647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:30.967137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] PATH_INFO: `/v3/projects/8b0c45536f43454eb65129d052a30ffd/users/96c2186a1d27432e8e831c999d91f365/roles/6cb27cd3219a4df396848116d7e9df7e` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:30.970434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0c45536f43454eb65129d052a30ffd, user_id=96c2186a1d27432e8e831c999d91f365, role_id=6cb27cd3219a4df396848116d7e9df7e)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:30.972368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:30.995052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 62/135] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:30 2022] PUT /identity/v3/projects/8b0c45536f43454eb65129d052a30ffd/users/96c2186a1d27432e8e831c999d91f365/roles/6cb27cd3219a4df396848116d7e9df7e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:31.058666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:31.086027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1ccf1-125a-4e8c-8f00-fbebc351ee94 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:31.138601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcd94a-e8c6-4b3c-aa27-2b2f02a338ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:31.139510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcd94a-e8c6-4b3c-aa27-2b2f02a338ee None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:31.139799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcd94a-e8c6-4b3c-aa27-2b2f02a338ee None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:31.140088 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcd94a-e8c6-4b3c-aa27-2b2f02a338ee None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:31.141418 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcd94a-e8c6-4b3c-aa27-2b2f02a338ee None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:31.143317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcd94a-e8c6-4b3c-aa27-2b2f02a338ee None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:31.145182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 74/136] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:30 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:31.151170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:31.155434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:31.289145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f8f34-ea1f-46da-824f-e262e630c67d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:31.340154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca60ae-e96e-46b1-a68f-6178dc39d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:31.341003 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca60ae-e96e-46b1-a68f-6178dc39d8cd None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:31.341322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca60ae-e96e-46b1-a68f-6178dc39d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:31.341613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca60ae-e96e-46b1-a68f-6178dc39d8cd None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/96c2186a1d27432e8e831c999d91f365/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:31.346786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca60ae-e96e-46b1-a68f-6178dc39d8cd None admin] RBAC: Authorizing `identity:create_grant(project_id=ac748ce4db4449de9948ac8e1f93cc8b, user_id=96c2186a1d27432e8e831c999d91f365, role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:31.348323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca60ae-e96e-46b1-a68f-6178dc39d8cd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:31.384247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 63/137] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:31 2022] PUT /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/96c2186a1d27432e8e831c999d91f365/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:31.385527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30165382-17cb-4e84-b352-e855a409e0fc None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:31.385825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30165382-17cb-4e84-b352-e855a409e0fc None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:31.386108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30165382-17cb-4e84-b352-e855a409e0fc None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:31.387639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 75/138] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:31.403346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:31.403688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:31.403938 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:31.425722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:31.427035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:31.498184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:31.509501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 64/139] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:31.545291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30165382-17cb-4e84-b352-e855a409e0fc None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:31.690177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30165382-17cb-4e84-b352-e855a409e0fc None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:31.729405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5c02a-36d8-4db7-b81a-d01670f5f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:31.730287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5c02a-36d8-4db7-b81a-d01670f5f674 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:31.730573 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5c02a-36d8-4db7-b81a-d01670f5f674 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:31.730856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5c02a-36d8-4db7-b81a-d01670f5f674 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:31.732054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5c02a-36d8-4db7-b81a-d01670f5f674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:31.733634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5c02a-36d8-4db7-b81a-d01670f5f674 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:31.739728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 76/140] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:31.740976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:31.857073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316dc54-3046-49bf-bb64-456f0c848983 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:31.897862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c31d57-7542-4ce2-8f6f-4f0363a27207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:31.898559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c31d57-7542-4ce2-8f6f-4f0363a27207 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:31.898852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c31d57-7542-4ce2-8f6f-4f0363a27207 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:31.899138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c31d57-7542-4ce2-8f6f-4f0363a27207 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:31.899886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c31d57-7542-4ce2-8f6f-4f0363a27207 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:31.901408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c31d57-7542-4ce2-8f6f-4f0363a27207 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:31.927071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 65/141] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:31 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:32.593378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb231f2-a1f6-41b3-b033-616b5fe53625 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:32.593894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb231f2-a1f6-41b3-b033-616b5fe53625 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:32.593894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb231f2-a1f6-41b3-b033-616b5fe53625 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:32.595495 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 77/142] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:32.601075 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:32.601417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:32.601706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:32.621787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:32.623001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:32.683497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:32.708707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 66/143] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:32.745317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb231f2-a1f6-41b3-b033-616b5fe53625 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:32.766833 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:32.870247 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb231f2-a1f6-41b3-b033-616b5fe53625 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:32.919001 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60333014-247f-466e-b464-c6e08f724492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:32.919835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60333014-247f-466e-b464-c6e08f724492 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:32.920119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60333014-247f-466e-b464-c6e08f724492 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:32.920403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60333014-247f-466e-b464-c6e08f724492 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:32.933741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60333014-247f-466e-b464-c6e08f724492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-60333014-247f-466e-b464-c6e08f724492 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:32.941632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:32.942674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:32.943616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 78/144] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:32.950814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:33.072897 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc2a18-26e4-49ab-90db-c17cda1d4dd9 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:33.112808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:33.113525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.113828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.114108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.114890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:33.115204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:33.116612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:33.126276 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 67/145] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:33.132002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60333014-247f-466e-b464-c6e08f724492 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:33.253044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:33.293335 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60333014-247f-466e-b464-c6e08f724492 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:33.334007 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923cda1-6bc0-4761-9543-4aa7de404542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:33.334922 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923cda1-6bc0-4761-9543-4aa7de404542 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.335212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923cda1-6bc0-4761-9543-4aa7de404542 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.335491 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923cda1-6bc0-4761-9543-4aa7de404542 None admin] PATH_INFO: `/v3/users/d7710a2b5d1c4650a14a9c8cb5d43b71` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.336794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923cda1-6bc0-4761-9543-4aa7de404542 None admin] RBAC: Authorizing `identity:get_user(user_id=d7710a2b5d1c4650a14a9c8cb5d43b71)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:33.338344 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923cda1-6bc0-4761-9543-4aa7de404542 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:33.340000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 79/146] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:33 2022] GET /identity/v3/users/d7710a2b5d1c4650a14a9c8cb5d43b71 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:33.346148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:33.424620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:33.488416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87c03-9b61-4f40-81a4-ac8ce85b9b61 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:33.532848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:33.533782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.534066 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.534344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.534871 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fef17-10b5-4e2b-86c6-094a5c618e19 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.535173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fef17-10b5-4e2b-86c6-094a5c618e19 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.535478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fef17-10b5-4e2b-86c6-094a5c618e19 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.535661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:33.537031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 80/147] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:33.537254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:33.538880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 68/148] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:33 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:33.543928 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.544198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.544489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.548425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:33.574689 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:33.575493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:33.670649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30662b76-22f8-40c9-8c85-d680a0bfa835 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:33.685207 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:33.689329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 81/149] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:33.723245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:33.724156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.724499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.724725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/d7710a2b5d1c4650a14a9c8cb5d43b71/roles/f89d9bb93fd142f4bb72e322d8502a85` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.730773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac748ce4db4449de9948ac8e1f93cc8b, user_id=d7710a2b5d1c4650a14a9c8cb5d43b71, role_id=f89d9bb93fd142f4bb72e322d8502a85)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:33.731571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:33.732264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:33.761476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 69/150] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:33 2022] PUT /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/users/d7710a2b5d1c4650a14a9c8cb5d43b71/roles/f89d9bb93fd142f4bb72e322d8502a85 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:33.839815 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:33.869756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b55f3-8e9c-4268-aae5-9f7347c13e1a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:33.918069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:33.918915 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:33.919201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:33.919486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:33.949355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:33.951461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:33.953856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:33.956816 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 82/151] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:33.963311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:34.147780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e9604-141e-4958-8856-93f0a77cec71 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:34.189672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:34.190367 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:34.190661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:34.190945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:34.191730 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:34.192034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:34.193495 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:34.204024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 70/152] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:34.209106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:34.364446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868b6db-0a67-4ed5-8920-632af32628cd None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:34.408391 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a30ce0-ce0d-4d08-8a30-1fe25ced898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:34.409331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a30ce0-ce0d-4d08-8a30-1fe25ced898d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:34.409621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a30ce0-ce0d-4d08-8a30-1fe25ced898d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:34.409887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a30ce0-ce0d-4d08-8a30-1fe25ced898d None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:34.411148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a30ce0-ce0d-4d08-8a30-1fe25ced898d None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:34.412669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a30ce0-ce0d-4d08-8a30-1fe25ced898d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:34.414384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 83/153] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:34 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:34.420187 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:34.467250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:34.545441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a3cd-3cb6-461f-857b-c61fde5ad01c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:34.588782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:34.589477 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:34.589767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:34.590040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:34.602916 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd28ad-5822-43b1-a744-9d5fa4a9cf21 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:34.603222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd28ad-5822-43b1-a744-9d5fa4a9cf21 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:34.603498 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd28ad-5822-43b1-a744-9d5fa4a9cf21 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:34.605055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 84/154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:34.610584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:34.610874 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:34.611152 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:34.625111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:34.648141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:34.649435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:34.694602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:34.696082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:34.757678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:34.810620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 85/155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:34.858475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:34.894234 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 71/156] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:34.900108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:35.095759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268f5b-a0c9-458e-9c99-dc300d6367f0 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:35.113404 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa341-18a6-4dfe-b2f0-78f8d5edabd6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:35.159030 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:35.166519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:35.167423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.167856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.168245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.169752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:35.171366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:35.176646 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 72/157] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:34 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:35.182873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:35.205987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b84c6-3758-4a0c-81d3-829c893b4674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:35.206829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b84c6-3758-4a0c-81d3-829c893b4674 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.207114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b84c6-3758-4a0c-81d3-829c893b4674 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.207391 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b84c6-3758-4a0c-81d3-829c893b4674 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.209604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:35.239947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b84c6-3758-4a0c-81d3-829c893b4674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-4c5b84c6-3758-4a0c-81d3-829c893b4674 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:35.248377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:35.249675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:35.250879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:35.251749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 86/158] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:35.252423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ea54d-6ed1-4b51-bea1-ea1234920db1 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.252784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ea54d-6ed1-4b51-bea1-ea1234920db1 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.253070 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ea54d-6ed1-4b51-bea1-ea1234920db1 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.254422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 87/159] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:35.262812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ea54d-6ed1-4b51-bea1-ea1234920db1 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:35.426739 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561041e1-2d3b-4834-bcfb-522649ef6a30 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:35.481267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:35.481267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.481267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.481267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.481754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=96c2186a1d27432e8e831c999d91f365) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:35.481754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:35.481754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cbe75-db82-47f9-99dd-fdb6475dd0ca None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:35.490084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 73/160] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:35 2022] GET /identity/v3/role_assignments?user.id=96c2186a1d27432e8e831c999d91f365&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:35.491415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.491704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.491983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.522554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:35.523816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:35.553884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ea54d-6ed1-4b51-bea1-ea1234920db1 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:35.638826 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:35.639942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.640442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.640947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.642423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:35.643010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:35.644674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211da899-b285-4c3f-9a57-fc0896899781 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:35.655329 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:35.665931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 88/161] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:35.667573 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ba428-0409-4e3c-9dbe-2d050e867872 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.667963 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ba428-0409-4e3c-9dbe-2d050e867872 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.668339 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ba428-0409-4e3c-9dbe-2d050e867872 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.673043 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 74/162] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:35.674764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 89/163] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:35.675232 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:35.680710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.680967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.681193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.714344 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:35.715584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:35.833495 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:35.874142 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f5d19-008d-421a-9c92-02c2335ddfeb None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:35.875034 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 90/164] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:35.876153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:35.952696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:35.953679 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:35.953972 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:35.954270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] PATH_INFO: `/v3/users/c7ff97c0da314994a8835ccf5732ab5e` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:35.957929 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] RBAC: Authorizing `identity:get_user(user_id=c7ff97c0da314994a8835ccf5732ab5e)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:35.959613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:35.961460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 75/165] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:35 2022] GET /identity/v3/users/c7ff97c0da314994a8835ccf5732ab5e => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:35.962355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:36.100006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:36.147433 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a62d1-6ee5-460d-b58e-0acd9fbe0c7e None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:36.187543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b63ed-9d05-4c67-a2e9-e94d3e35e182 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:36.250071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:36.251341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.251842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.252335 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.282364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:36.289271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:36.289271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.289271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.289271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:36.295092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:36.296203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:36.297159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:36.299791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 91/166] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:36.300847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:36.322461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:36.335525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:36.336706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:36.337662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 76/167] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:01:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:36.338542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:36.410351 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:36.498006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeacf7-1dbc-455e-88c7-65bc4d503be2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:36.531912 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:36.545918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:36.547280 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.547851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.548298 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.549418 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:36.549885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:36.551772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:36.555043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4e8b0-4159-43f1-a167-d732aff0bb50 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:36.561794 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 77/168] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:36.563747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:36.653906 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956db7a-10eb-4ae7-857b-d8a807dc6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:36.654494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956db7a-10eb-4ae7-857b-d8a807dc6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.654784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956db7a-10eb-4ae7-857b-d8a807dc6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.655063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956db7a-10eb-4ae7-857b-d8a807dc6aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.673625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5c73c-a286-4757-8bcd-0322d10153a1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:36.712745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308985e2-7831-4489-b365-9d6031304eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:36.713753 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308985e2-7831-4489-b365-9d6031304eda None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.714461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308985e2-7831-4489-b365-9d6031304eda None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.714898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308985e2-7831-4489-b365-9d6031304eda None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.715967 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308985e2-7831-4489-b365-9d6031304eda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:36.716457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308985e2-7831-4489-b365-9d6031304eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:36.718373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308985e2-7831-4489-b365-9d6031304eda None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:36.728552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 78/169] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:01:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:36.730153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5ef0-4046-49b9-b59c-ce28b5e511fb None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.730607 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5ef0-4046-49b9-b59c-ce28b5e511fb None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.731065 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5ef0-4046-49b9-b59c-ce28b5e511fb None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.732524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 79/170] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:36.733405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c5ef0-4046-49b9-b59c-ce28b5e511fb None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:36.792134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:36.805381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956db7a-10eb-4ae7-857b-d8a807dc6aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:36.806923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956db7a-10eb-4ae7-857b-d8a807dc6aa8 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:36.870798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c5ef0-4046-49b9-b59c-ce28b5e511fb None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:36.896997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:36.940896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:36.941838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.942128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.942410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:36.943659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:36.945133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:36.947175 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 80/171] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:36 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:36.947979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:36.961480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 92/172] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:36.962761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:36.963062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:36.963344 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.010545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:37.011787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:37.079588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d7168-4211-4ceb-9480-f82149073741 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.104501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 93/173] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:37.105446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.109123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.154934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.155830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.156114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.156393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] PATH_INFO: `/v3/users/96c2186a1d27432e8e831c999d91f365` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.157944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2186a1d27432e8e831c999d91f365)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:37.159422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:37.165870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 81/174] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:36 2022] GET /identity/v3/users/96c2186a1d27432e8e831c999d91f365 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:37.166732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.285419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.340146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52333-a583-4129-adda-7074cbf09ca3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.381254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.382154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.382441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.382570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bce58-16f5-4c52-bb07-a192f2b91630 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.382782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.383963 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:37.385722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:37.387745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 82/175] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:37.388356 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.450301 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.454854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.455610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.455951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.456256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.510435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df370866-1055-428a-bacd-eab6800bb250 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.528855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:37.530394 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:37.546287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.547104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.547451 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.547730 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.558097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:37.565902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:37.567086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:37.568021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 83/176] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:37.568785 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.613629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.639919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 94/177] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:37.640787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.680861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2366a2-b798-455e-825f-ee3e2f159983 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.691724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.718084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.718767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.719066 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.719347 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.751324 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157ccb-cfc1-4737-81b5-ebed09ed10eb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.786575 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.787238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.787522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.787796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.788639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=c7ff97c0da314994a8835ccf5732ab5e) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:37.788949 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:37.790519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9987d-4c23-45c3-a09e-cd0857354b7b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:37.791464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:37.792919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:37.796692 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 95/178] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/role_assignments?user.id=c7ff97c0da314994a8835ccf5732ab5e&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:37.809494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aa334-a81d-4013-8532-5f640312287e None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.809494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aa334-a81d-4013-8532-5f640312287e None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.809494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aa334-a81d-4013-8532-5f640312287e None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.809494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 96/179] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:37.809494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77f3f-3e21-41e6-bee6-1dcd67d4d917 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.809494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77f3f-3e21-41e6-bee6-1dcd67d4d917 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.809938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77f3f-3e21-41e6-bee6-1dcd67d4d917 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.809938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 97/180] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:37.809938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb7d30-bbbf-4450-99b1-d99a23b21cd4 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.809938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb7d30-bbbf-4450-99b1-d99a23b21cd4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.809938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb7d30-bbbf-4450-99b1-d99a23b21cd4 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.814509 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 98/181] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:37.814509 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb7d30-bbbf-4450-99b1-d99a23b21cd4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.864169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.912888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 84/182] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:37.918420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:37.933378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb7d30-bbbf-4450-99b1-d99a23b21cd4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:37.934001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:37.973768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:37.974575 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.974863 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.975145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:37.975924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:37.976238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:37.977712 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f2d8-4450-4347-abe1-1f4dbb0837f3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:37.991281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 99/183] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:37.993034 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f62eb7-afc1-47e3-9f89-397e68c24d66 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:37.993385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f62eb7-afc1-47e3-9f89-397e68c24d66 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:37.993630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f62eb7-afc1-47e3-9f89-397e68c24d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.011680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-e7f62eb7-afc1-47e3-9f89-397e68c24d66 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:38.012885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-e7f62eb7-afc1-47e3-9f89-397e68c24d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:38.056580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee19a-81a2-409f-9bff-257a04b43f68 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.094921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 100/184] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:38.096241 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.096533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.096813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.103441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ce583-c3b9-429b-9a17-a80280e5ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.104315 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ce583-c3b9-429b-9a17-a80280e5ed15 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.104601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ce583-c3b9-429b-9a17-a80280e5ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.104870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ce583-c3b9-429b-9a17-a80280e5ed15 None admin] PATH_INFO: `/v3/projects/8b0c45536f43454eb65129d052a30ffd` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.106117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ce583-c3b9-429b-9a17-a80280e5ed15 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0c45536f43454eb65129d052a30ffd)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.110034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ce583-c3b9-429b-9a17-a80280e5ed15 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.112047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 85/185] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:37 2022] GET /identity/v3/projects/8b0c45536f43454eb65129d052a30ffd => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:38.113066 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783678-70f2-478f-bb49-317e076e203e None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.113468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783678-70f2-478f-bb49-317e076e203e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.113785 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783678-70f2-478f-bb49-317e076e203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.118258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:38.119407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:38.132360 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-8a783678-70f2-478f-bb49-317e076e203e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:38.133577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-8a783678-70f2-478f-bb49-317e076e203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:38.146482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.189535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 101/186] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:38.190376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.202316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 86/187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:38.203707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a783678-70f2-478f-bb49-317e076e203e None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.225271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.291240 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae95c5-c13a-44ee-8148-aa9e69ef28ed None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.317979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a783678-70f2-478f-bb49-317e076e203e None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.324053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.324938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.325218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.325517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.326713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.328238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.330100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 102/188] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:38.330876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.354217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.355097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.355502 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.355641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] PATH_INFO: `/v3/users/72fd2006a5f046a9860bb25536fbba37` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.363209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.371427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd2006a5f046a9860bb25536fbba37)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.372838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.374735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 87/189] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/users/72fd2006a5f046a9860bb25536fbba37 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:38.375339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.438396 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcd0e-f710-4248-8a9e-a05a71472572 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.439888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.473978 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.474647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.474755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acfb88-43d7-45ab-8e83-2ba258562522 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.474934 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.475211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.476054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=8b0c45536f43454eb65129d052a30ffd, user.id=96c2186a1d27432e8e831c999d91f365) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:38.476371 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.477950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.483765 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 103/190] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/role_assignments?user.id=96c2186a1d27432e8e831c999d91f365&scope.project.id=8b0c45536f43454eb65129d052a30ffd&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:38.484601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.525270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.526126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.526407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.526681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.527993 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.529517 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.531251 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 88/191] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:38.532409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.594440 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.603326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130738-2a26-4ccb-99cc-5b8e1aa357e8 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.643043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.643781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.644070 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.644346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.647756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb3187-b94b-49da-8d0f-52360b0137e8 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.656169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:38.664226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:38.665412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:38.666543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:38.670500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 104/192] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:38.671382 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.691304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.692154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.692379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.692666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.694731 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.707339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:38.715815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:38.716842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:38.717856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:38.719074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 89/193] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:38.719532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.786434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c1769-6b34-45e7-91aa-fff2d5675b12 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.819368 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.820040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.820330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.820609 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.821486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] RBAC: Adding query filter params (role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=96c2186a1d27432e8e831c999d91f365) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:38.821807 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.823300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.829405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de0a4c-3fcc-4f3a-9ac5-67e240d445a7 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.829606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 105/194] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/role_assignments?user.id=96c2186a1d27432e8e831c999d91f365&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:38.834085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:38.856035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.868894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.869610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.869896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.870170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.940964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:38.951932 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.952926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1882adf-a2e0-44c6-a8e6-0db7bcab37bc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:38.953650 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:38.991761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:38.992458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:38.992739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:38.993015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:38.993801 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:38.995303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.026322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 106/195] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:38 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:39.027184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.085223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 90/196] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:39.085999 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.135759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:39.169746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1827f5-f311-488e-a5f3-6d7fdd74a4ed None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.206378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:39.207753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.208530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.208814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.209183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.210059 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:39.210370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.211817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.215957 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1972a-d9d8-47c1-99c1-35e963772301 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.221844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 107/197] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:39.221844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.262830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.263769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.264061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.264341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.265314 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:39.265535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.273324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.276676 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 91/198] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:39.277180 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.356768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df8317-ce21-4271-8d25-b0ff77b694b4 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.409571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.410483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.410750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.411025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.412977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.414574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.416672 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 108/199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:39.417502 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.423927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe90ede-0fdd-4238-88d0-5332b259a3bd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.427523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:39.472398 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.474793 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.475756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.476282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] PATH_INFO: `/v3/users/d7710a2b5d1c4650a14a9c8cb5d43b71` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.477852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] RBAC: Authorizing `identity:get_user(user_id=d7710a2b5d1c4650a14a9c8cb5d43b71)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.481868 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.485277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 92/200] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/users/d7710a2b5d1c4650a14a9c8cb5d43b71 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:39.486727 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.520011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:39.538808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6d5ae-eabb-42d7-b7ab-30785f91e293 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.576789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.577885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.578240 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.578630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] PATH_INFO: `/v3/users/72fd2006a5f046a9860bb25536fbba37` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.579941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd2006a5f046a9860bb25536fbba37)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.581546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.583309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 109/201] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/users/72fd2006a5f046a9860bb25536fbba37 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:39.584275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.618948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f417c8-3f82-4312-8336-828d91bfd175 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.665626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:39.669054 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.670035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.670551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.671063 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.673173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=72fd2006a5f046a9860bb25536fbba37) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:39.673780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.677000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.686377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 93/202] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/role_assignments?user.id=72fd2006a5f046a9860bb25536fbba37&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:39.687334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:39.699988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66e8a-0c7d-412f-986e-99265f50dbe1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.764971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.765647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.766036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.766324 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:39.876125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:39.878195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:39.882457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:39.904093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74481984-0c57-4a5b-bc4c-15e6be344cae None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:39.964333 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:39.965006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:39.965318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:39.965602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:40.007755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:40.044222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 110/203] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:40.049880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:40.096566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:40.098112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:40.193178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:40.207923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 94/204] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:40.223196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbbe89-0e75-4e74-b24a-2a9c55f4ca96 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:40.225815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:40.306047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:40.306859 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:40.307138 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:40.307407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:40.308658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:40.310176 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:40.311897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 111/205] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:40 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:40.318003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:40.336648 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012b513-cefd-43b2-b6cd-561158fe160c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:40.388583 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:40.389517 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:40.389796 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:40.390071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:40.393618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:40.393772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:40.395167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:40.397125 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 95/206] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:40 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:40.407193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:40.495783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21fdc-a238-462d-a0b3-fa5c2b865306 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:40.518485 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:40.573619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39e996-f0e8-40b3-966a-68aa1c89a9ae None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:40.575317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:40.576003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:40.576291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:40.576567 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:40.577506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] RBAC: Adding query filter params (role.id=f89d9bb93fd142f4bb72e322d8502a85, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b, user.id=d7710a2b5d1c4650a14a9c8cb5d43b71) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:40.577824 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:40.579494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5f4c-0c39-41af-9887-31ee3a8b1054 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:40.596478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 112/207] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:40 2022] GET /identity/v3/role_assignments?user.id=d7710a2b5d1c4650a14a9c8cb5d43b71&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=f89d9bb93fd142f4bb72e322d8502a85 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:40.610919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:40.611705 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:40.612049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:40.612438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:40.613444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] RBAC: Adding query filter params (role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=72fd2006a5f046a9860bb25536fbba37) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:40.613865 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:40.615465 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3e1f9-8cab-4cbb-82c2-ad7356f98894 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:40.622320 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 96/208] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:40 2022] GET /identity/v3/role_assignments?user.id=72fd2006a5f046a9860bb25536fbba37&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:41.560185 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee20e5-408c-45c9-9f7d-feae72cd7419 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:41.560737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee20e5-408c-45c9-9f7d-feae72cd7419 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:41.560737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee20e5-408c-45c9-9f7d-feae72cd7419 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:41.562077 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 113/209] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:41.567582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:41.568054 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:41.568142 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:41.585447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:41.586651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:41.655868 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2cdc8-0108-4675-8355-6b33f74cf491 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:41.656205 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2cdc8-0108-4675-8355-6b33f74cf491 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:41.656681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2cdc8-0108-4675-8355-6b33f74cf491 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:41.659588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 114/210] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:41.664297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955542-b1e0-4819-a716-c0746d252803 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:41.664656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955542-b1e0-4819-a716-c0746d252803 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:41.665012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955542-b1e0-4819-a716-c0746d252803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:41.667455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 97/211] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:41.682734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-22955542-b1e0-4819-a716-c0746d252803 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:41.684038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-22955542-b1e0-4819-a716-c0746d252803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:41.689984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:41.742546 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:41.772347 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 115/212] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:41.796940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955542-b1e0-4819-a716-c0746d252803 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:41.804689 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6176e-7eb5-4f3c-bc83-5e6386ae025b None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:41.833751 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:41.841444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:41.843470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:41.843986 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:41.844511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:41.855967 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:41.864619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:41.865900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:41.867112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:41.869716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 98/213] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:41.875568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:41.906080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955542-b1e0-4819-a716-c0746d252803 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:41.943211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:41.944354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:41.944714 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:41.945064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:41.947409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:41.949067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:41.953374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 116/214] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:41.958412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:41.993918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1ce79-80d0-42ab-93a0-08b87a4a6ba5 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.007320 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.037021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.038150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.038664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.039171 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.040101 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:42.040590 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:42.042258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:42.052218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 99/215] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:42.058919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:42.070917 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c8963-f122-4d96-b1db-3fd4540d0808 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.072635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.110453 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee5e96-9fc9-4f81-92d3-952b67266229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.111274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee5e96-9fc9-4f81-92d3-952b67266229 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.111628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee5e96-9fc9-4f81-92d3-952b67266229 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.111977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee5e96-9fc9-4f81-92d3-952b67266229 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.112783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee5e96-9fc9-4f81-92d3-952b67266229 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:42.114434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee5e96-9fc9-4f81-92d3-952b67266229 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:42.145322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 117/216] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:41 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:42.193586 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d9ee4-00b5-4fd4-bd5e-9f78a8e6676b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.195700 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f84e1-1618-444b-a543-e75d7edf7eb4 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.195997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f84e1-1618-444b-a543-e75d7edf7eb4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.196278 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f84e1-1618-444b-a543-e75d7edf7eb4 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.198023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 118/217] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:01:42.203614 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.203901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.204183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.225080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:42.226589 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:42.236397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.237349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.237639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.237919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] PATH_INFO: `/v3/users/61218c92925543e8b9ae7dcee110e605` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.254360 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] RBAC: Authorizing `identity:get_user(user_id=61218c92925543e8b9ae7dcee110e605)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:42.255951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:42.263010 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 100/218] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/users/61218c92925543e8b9ae7dcee110e605 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:42.265267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:42.304529 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.337668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 119/219] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:42.354924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:42.391574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33627eb4-a57a-48f4-91ef-ef4bdf3a95f9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.439581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.440639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.440934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.441270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.466306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.478649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08217c7d-cdcd-41d3-a4fc-3f8af50f2ab8 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.521983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.522830 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.523108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.523387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.529689 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:42.532090 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:42.540596 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:42.548600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:42.549731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:42.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 120/220] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:42.555649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:42.570283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.672584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 101/221] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:42.679966 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:42.688983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91bba9-a899-4dfe-82cd-73e9156f7c58 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.746120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.746897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.747184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.747461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.748336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:42.748660 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:42.750363 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:42.762500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 121/222] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:42.775246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:42.794019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.818105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823077-6cf7-4ead-8fc4-fe8c26d78cd1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:42.849958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:42.858645 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:42.859536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:42.859931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:42.860214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:42.863628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:42.863628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:42.865516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 102/223] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:42.871315 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:43.001476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500ec11-6ae0-4b20-bae1-02e4c085aab0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.017991 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff8585-d304-4849-a84a-7750f55bf9c3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.074835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.075742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.076030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.076310 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] PATH_INFO: `/v3/users/72fd2006a5f046a9860bb25536fbba37` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.077692 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd2006a5f046a9860bb25536fbba37)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:43.079239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:43.080924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 122/224] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/users/72fd2006a5f046a9860bb25536fbba37 => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:43.084844 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.085545 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.085850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.086122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.087218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:43.088848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=61218c92925543e8b9ae7dcee110e605) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:43.089176 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:43.090690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc3a00-4232-4f09-912b-2e07bda0f7b2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:43.105819 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 103/225] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:42 2022] GET /identity/v3/role_assignments?user.id=61218c92925543e8b9ae7dcee110e605&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:43.182038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:43.194747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622305bf-319e-4953-b2e9-24595fb1d079 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.195082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622305bf-319e-4953-b2e9-24595fb1d079 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.195316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622305bf-319e-4953-b2e9-24595fb1d079 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.197027 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 104/226] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:01:43.202723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.203034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.203344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.216857 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34ee67-d6f1-4124-ab7d-83c2e4eaae47 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.224120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:43.225405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:43.302961 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.303869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.304155 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.304435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.306640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:43.308177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:43.312544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 123/227] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:43 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:43.316309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:43.345625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:43.367490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 105/228] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:43.390578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:43.448623 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aeb90-10c4-490d-9e7d-a60065c5d8fa None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.492211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.493132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.493447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.493723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/72fd2006a5f046a9860bb25536fbba37/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.497286 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user_id=72fd2006a5f046a9860bb25536fbba37, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:43.498767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:43.523916 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 124/229] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:43 2022] PUT /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/72fd2006a5f046a9860bb25536fbba37/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 15:01:43.568723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1238c-2d5a-45fe-829b-91d7156ccc62 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.606317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.607271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.607612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.607956 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.619626 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:43.628266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:43.629413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:43.630343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 106/230] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:01:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:43.637747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:43.649757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:43.752393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545a8a7-2b55-4ffc-90fc-b9c0fddb0a96 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.790051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.790657 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.790942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.791222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.791997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:43.792306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:43.793753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:43.807622 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 125/231] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:01:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:43.809039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:43.868076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:43.920311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed77aca-151d-4cb1-b2e8-480921fc6607 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:43.956803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:43.957770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:43.958102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:43.958386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] PATH_INFO: `/v3/users/72fd2006a5f046a9860bb25536fbba37` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:43.959628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd2006a5f046a9860bb25536fbba37)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:43.961095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:43.962818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 107/232] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:43 2022] GET /identity/v3/users/72fd2006a5f046a9860bb25536fbba37 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:43.970463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:44.050207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:44.085071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059bf5d-2d92-40b9-b3fa-c07307fcffee None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:44.120610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72e71c-ea6c-476f-af26-69fee99a0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:44.121533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72e71c-ea6c-476f-af26-69fee99a0286 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:44.121819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72e71c-ea6c-476f-af26-69fee99a0286 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:44.122097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72e71c-ea6c-476f-af26-69fee99a0286 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:44.123303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72e71c-ea6c-476f-af26-69fee99a0286 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:44.124806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72e71c-ea6c-476f-af26-69fee99a0286 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:44.130827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 126/233] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:43 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:44.132807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:44.263008 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156be6d7-8a31-4b92-99e8-3dae40a363d9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:44.286483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:44.302889 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cdd13-a5db-4c6d-baf0-2977a7b33565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:44.304089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cdd13-a5db-4c6d-baf0-2977a7b33565 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:44.304624 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cdd13-a5db-4c6d-baf0-2977a7b33565 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:44.305092 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cdd13-a5db-4c6d-baf0-2977a7b33565 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/72fd2006a5f046a9860bb25536fbba37/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:44.307674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cdd13-a5db-4c6d-baf0-2977a7b33565 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user_id=72fd2006a5f046a9860bb25536fbba37, role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:44.309369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cdd13-a5db-4c6d-baf0-2977a7b33565 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:44.336165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 108/234] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:44 2022] PUT /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/72fd2006a5f046a9860bb25536fbba37/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:45.169222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277de4-7c05-4911-a7d5-824d6e4a4580 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.170200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277de4-7c05-4911-a7d5-824d6e4a4580 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.170679 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277de4-7c05-4911-a7d5-824d6e4a4580 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.172801 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 127/235] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:45.178189 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac1713-5dee-4d74-b1d0-c2d2cc215702 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.179091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac1713-5dee-4d74-b1d0-c2d2cc215702 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.179565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac1713-5dee-4d74-b1d0-c2d2cc215702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.202688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-c1ac1713-5dee-4d74-b1d0-c2d2cc215702 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:45.204267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-c1ac1713-5dee-4d74-b1d0-c2d2cc215702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:45.298173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 109/236] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:45.324668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8277de4-7c05-4911-a7d5-824d6e4a4580 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:45.409490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce833e43-e6fa-4646-b44f-1701a59a4876 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.410120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce833e43-e6fa-4646-b44f-1701a59a4876 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.410603 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce833e43-e6fa-4646-b44f-1701a59a4876 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.412249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 110/237] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:01:45.417569 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70298c10-901b-411c-974c-11a8997728ae None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.418117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70298c10-901b-411c-974c-11a8997728ae None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.418587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70298c10-901b-411c-974c-11a8997728ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.449424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-70298c10-901b-411c-974c-11a8997728ae None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:45.450950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-70298c10-901b-411c-974c-11a8997728ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:45.475426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8277de4-7c05-4911-a7d5-824d6e4a4580 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:45.512570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:45.513745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.514280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.514773 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.518217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:45.527952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:45.536307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:45.537411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:45.538395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:45.541388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 128/238] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:45.553876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:45.565216 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 111/239] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 15:01:45.604139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70298c10-901b-411c-974c-11a8997728ae None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:45.660762 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47550f-9787-40d2-b1b1-06b5e64e16a7 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:45.701671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:45.704174 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.704720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.705218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.706273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:45.706822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:45.708523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:45.719551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 129/240] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:45.723754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:45.725212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:45.832394 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70298c10-901b-411c-974c-11a8997728ae None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:45.851317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:45.870280 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:45.871536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.872104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.872644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.874115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:45.875857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:45.878161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 112/241] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:45.883792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:45.942562 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365860a-6394-4f21-9b0a-ba2dee2c75fa None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:45.989866 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42af86-a793-45c5-af47-cc40ed459c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:45.991687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42af86-a793-45c5-af47-cc40ed459c2d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:45.992147 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42af86-a793-45c5-af47-cc40ed459c2d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:45.992830 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42af86-a793-45c5-af47-cc40ed459c2d None admin] PATH_INFO: `/v3/users/868d258fe9ec4af7abde27bad45fef91` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:45.998280 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72978ca-3677-4779-bb3b-00ac038aea39 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:46.011201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:46.012132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42af86-a793-45c5-af47-cc40ed459c2d None admin] RBAC: Authorizing `identity:get_user(user_id=868d258fe9ec4af7abde27bad45fef91)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:46.013925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42af86-a793-45c5-af47-cc40ed459c2d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:46.017487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 130/242] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:45 2022] GET /identity/v3/users/868d258fe9ec4af7abde27bad45fef91 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:46.019214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91db13d-8fa6-4c85-a9a3-35ae57eeecf6 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.019873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91db13d-8fa6-4c85-a9a3-35ae57eeecf6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.020171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91db13d-8fa6-4c85-a9a3-35ae57eeecf6 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.022093 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 131/243] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:46.022806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91db13d-8fa6-4c85-a9a3-35ae57eeecf6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:46.047760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd5727-a4c5-4826-bf5c-45035df8a15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:46.048818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5727-a4c5-4826-bf5c-45035df8a15c None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.049396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5727-a4c5-4826-bf5c-45035df8a15c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.050294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5727-a4c5-4826-bf5c-45035df8a15c None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.051584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5727-a4c5-4826-bf5c-45035df8a15c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:46.053980 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5727-a4c5-4826-bf5c-45035df8a15c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:46.081639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 113/244] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:45 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:46.083475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.084001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.084512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.105649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:46.107867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:46.212027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91db13d-8fa6-4c85-a9a3-35ae57eeecf6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:46.238752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:46.251020 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 114/245] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:46.253375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33384527-dc53-4712-8d6f-a5b5a068a19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:46.254342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33384527-dc53-4712-8d6f-a5b5a068a19e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.254847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33384527-dc53-4712-8d6f-a5b5a068a19e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.255316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33384527-dc53-4712-8d6f-a5b5a068a19e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.289733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:46.338212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33384527-dc53-4712-8d6f-a5b5a068a19e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:46.340185 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33384527-dc53-4712-8d6f-a5b5a068a19e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:46.415444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:46.427148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c2115-c815-4e3a-9719-c19f10276b20 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:46.459299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 132/246] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:46.462065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4be9-edc8-4fb0-bf0d-77e487cd6fc4 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.462065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4be9-edc8-4fb0-bf0d-77e487cd6fc4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.462065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4be9-edc8-4fb0-bf0d-77e487cd6fc4 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.462065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 133/247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:46.466964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a4be9-edc8-4fb0-bf0d-77e487cd6fc4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:46.480142 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429510e-3df3-4264-af4f-81c4c9fd75f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:46.480967 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429510e-3df3-4264-af4f-81c4c9fd75f1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.481280 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429510e-3df3-4264-af4f-81c4c9fd75f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.481562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429510e-3df3-4264-af4f-81c4c9fd75f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.492033 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:46.500471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429510e-3df3-4264-af4f-81c4c9fd75f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6429510e-3df3-4264-af4f-81c4c9fd75f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:46.508539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:46.509706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:46.510634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:46.512505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 115/248] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:46.514543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.514543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.514543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.537294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:46.539262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:46.616707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 116/249] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:46.617816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:46.622493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a4be9-edc8-4fb0-bf0d-77e487cd6fc4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:46.668785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:46.670927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.671599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.672358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.675292 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:46.677053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:46.680233 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 134/250] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:46.681523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:46.731698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:46.764027 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a489255-6fd7-4f41-a825-68d57d8da0a5 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:46.826128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:46.826823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.827111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.827566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.829091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:46.829679 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:46.831368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:46.843702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:46.852099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 117/251] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:46.853786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:46.856000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe09e0-ce2c-4a20-938b-2a45ba942d7e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:46.943406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:46.944236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:46.944513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:46.944788 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:46.964613 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:46.976789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:46.977910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:46.978844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 135/252] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:46.979604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:47.032516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f411991-8285-4b4f-a816-51e0408c4e99 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.098484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:47.099158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.099434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.099719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.100569 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] RBAC: Adding query filter params (role.id=f89d9bb93fd142f4bb72e322d8502a85, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=868d258fe9ec4af7abde27bad45fef91) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:47.100888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:47.102400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:47.110564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 118/253] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/role_assignments?user.id=868d258fe9ec4af7abde27bad45fef91&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=f89d9bb93fd142f4bb72e322d8502a85 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:47.111395 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:47.139095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:47.179105 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da377c7d-06e8-439c-a7c9-ede2da01e8e0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.251299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:47.252220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.252510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.252782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] PATH_INFO: `/v3/users/61218c92925543e8b9ae7dcee110e605` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.255710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] RBAC: Authorizing `identity:get_user(user_id=61218c92925543e8b9ae7dcee110e605)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:47.257267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:47.259098 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 136/254] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:46 2022] GET /identity/v3/users/61218c92925543e8b9ae7dcee110e605 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:47.271562 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:47.296029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:47.326686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115835-b6bb-4a3d-93de-2b7673146180 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.395419 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:47.396112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.396395 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.396673 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.397478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:47.397789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:47.399217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2154dd-f8f1-48f5-bb65-986bf75de3c8 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:47.415617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 119/255] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:47.417001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f85d-fc33-4da1-96d4-d2c02d22607e None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.417318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f85d-fc33-4da1-96d4-d2c02d22607e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.417600 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4f85d-fc33-4da1-96d4-d2c02d22607e None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.418873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 120/256] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:47.422553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f85d-fc33-4da1-96d4-d2c02d22607e None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:47.455420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145c96-8813-4073-a9b2-5093b0c251d5 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.528188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4a6d-c677-4fb7-a49d-7f80fd0d35df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:47.530385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4a6d-c677-4fb7-a49d-7f80fd0d35df None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.531183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4a6d-c677-4fb7-a49d-7f80fd0d35df None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.532044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:47.532779 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4a6d-c677-4fb7-a49d-7f80fd0d35df None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.535531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4a6d-c677-4fb7-a49d-7f80fd0d35df None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:47.538630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4a6d-c677-4fb7-a49d-7f80fd0d35df None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:47.543009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 137/257] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:47 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:47.545613 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.546654 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.547477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.578828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:47.581676 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:47.610479 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4f85d-fc33-4da1-96d4-d2c02d22607e None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.696604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:47.697610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.697888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.698220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] PATH_INFO: `/v3/users/72fd2006a5f046a9860bb25536fbba37` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.702135 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd2006a5f046a9860bb25536fbba37)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:47.704229 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:47.705963 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 121/258] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:47 2022] GET /identity/v3/users/72fd2006a5f046a9860bb25536fbba37 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:47.706793 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:47.720041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 138/259] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:47.722008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:47.746799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:47.927365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b55d9b-a11b-462a-841e-7ac6b1e98882 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.943639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32232d62-0414-4590-8e92-f3366667d3c4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:47.994468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:47.995390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:47.995836 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:47.996126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/61218c92925543e8b9ae7dcee110e605/roles/6cb27cd3219a4df396848116d7e9df7e` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:47.998848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] RBAC: Authorizing `identity:create_grant(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user_id=61218c92925543e8b9ae7dcee110e605, role_id=6cb27cd3219a4df396848116d7e9df7e)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:48.000348 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:48.010999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:48.011740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:48.012171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:48.012329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:48.028962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 122/260] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:47 2022] PUT /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/61218c92925543e8b9ae7dcee110e605/roles/6cb27cd3219a4df396848116d7e9df7e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:48.029774 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:48.031071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:48.163399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:48.164918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:48.189423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:48.231139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee268ae1-631e-44a1-bed8-1bc8b4c1732c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:48.284233 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:48.285080 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:48.285411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:48.285698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:48.305304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:48.323884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:48.325755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:48.328040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 123/261] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:01:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:48.332755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:48.377260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 139/262] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:48.385515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:48.428629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:48.471836 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b278e34-ccc3-44d2-8ad4-d653fddd1395 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:48.516442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:48.517927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:48.518425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:48.518917 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:48.520329 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:48.520899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:48.523558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:48.561410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 124/263] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:01:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:48.566478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:48.706817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a7315-da5d-4870-b146-fc12c497a2f2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:48.739213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:48.816493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eddec-15af-4374-8a87-b4fc36c287a1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:48.819860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:48.827942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:48.828683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:48.829180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:48.831516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:48.837749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:48.845472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 140/264] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:48 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:48.849684 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:48.851388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:48.869089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:48.870047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:48.870343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:48.870613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] PATH_INFO: `/v3/users/72fd2006a5f046a9860bb25536fbba37` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:48.872048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd2006a5f046a9860bb25536fbba37)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:48.873622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:48.876788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 125/265] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:48 2022] GET /identity/v3/users/72fd2006a5f046a9860bb25536fbba37 => generated 283 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:48.882502 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:49.058634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ffcc2-baa5-4132-8ba0-1ab1b5f9fc26 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:49.075413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317350-5493-4ba4-9380-8ba553c55219 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:49.128910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:49.129605 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.129889 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.130167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.131014 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=72fd2006a5f046a9860bb25536fbba37) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:49.131328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:49.132813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071930f1-1eb4-4949-8694-bcd77636ffec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:49.144550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:49.145212 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.145532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.145809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.147730 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 141/266] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:48 2022] GET /identity/v3/role_assignments?user.id=72fd2006a5f046a9860bb25536fbba37&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:49.159500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e740b52-7497-42f0-a275-743eb799ebf0 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.160297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e740b52-7497-42f0-a275-743eb799ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.160598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e740b52-7497-42f0-a275-743eb799ebf0 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.162424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 142/267] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:01:49.172641 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.173015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.173191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.210669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:49.211911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:49.268761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:49.288248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:49.290440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:49.301656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:49.372232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 143/268] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:49.417960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:49.486683 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 126/269] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:49.493338 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:49.630344 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb9d31-0827-4e45-8194-55bbacdf8099 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:49.674950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:49.691179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed68ad-6095-4efd-b475-7d50122fa44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:49.692014 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed68ad-6095-4efd-b475-7d50122fa44a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.692299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed68ad-6095-4efd-b475-7d50122fa44a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.692577 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed68ad-6095-4efd-b475-7d50122fa44a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.693306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb643a-e31e-4ee1-b676-4803469f3821 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:49.694078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed68ad-6095-4efd-b475-7d50122fa44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:49.695560 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed68ad-6095-4efd-b475-7d50122fa44a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:49.702599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 144/270] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:49.704403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d2a6-509f-4df7-a478-5162196e734c None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.704858 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d2a6-509f-4df7-a478-5162196e734c None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.705331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d2a6-509f-4df7-a478-5162196e734c None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.706836 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 145/271] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:49.713123 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.713618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.714078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.733579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:49.735111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:49.737205 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:49.738216 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.738507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.738785 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.740012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:49.741519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:49.749277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 127/272] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:49 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:49.749277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:49.778262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:49.821214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 146/273] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:49.822427 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:49.852932 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764ff30-1357-4381-ad1f-536b58193967 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:49.888907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:49.889580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:49.889863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:49.890143 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:49.890907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:49.892762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:49.921389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 128/274] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:01:49 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:49.921389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:49.935625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:49.991026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e099006-849a-4fd1-8182-cb0ac4a7d41e None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:50.044645 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f058f8-58d0-4e28-a5e5-70b779ae521a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:50.067544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:50.068230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.068511 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.068787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.069678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] RBAC: Adding query filter params (role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=72fd2006a5f046a9860bb25536fbba37) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:50.069991 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:50.071513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:50.086729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:50.087563 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.087850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.088128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.090970 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 147/275] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:49 2022] GET /identity/v3/role_assignments?user.id=72fd2006a5f046a9860bb25536fbba37&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:50.100767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:50.109102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:50.110326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:50.111697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 129/276] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:50.118756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:50.185782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:50.235752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3504fc-e1a2-4554-909f-54d871ed6eb6 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:50.286652 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:50.287365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.287652 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.287928 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.288713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] RBAC: Adding query filter params (name=member) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:50.289025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:50.290495 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:50.308163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 148/277] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:50.315315 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:50.399441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:50.430801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cce17-ab95-425b-9c10-2a1a276fbf62 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:50.473169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb7dea-3ca5-4625-aed1-704cf4214b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:50.474094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7dea-3ca5-4625-aed1-704cf4214b5f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.474381 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7dea-3ca5-4625-aed1-704cf4214b5f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.474661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7dea-3ca5-4625-aed1-704cf4214b5f None admin] PATH_INFO: `/v3/users/208021a9d5f74cc2813f19291070a807` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.491812 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb7dea-3ca5-4625-aed1-704cf4214b5f None admin] RBAC: Authorizing `identity:get_user(user_id=208021a9d5f74cc2813f19291070a807)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:50.493428 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb7dea-3ca5-4625-aed1-704cf4214b5f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:50.500083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 130/278] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:50 2022] GET /identity/v3/users/208021a9d5f74cc2813f19291070a807 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:50.501510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:50.614305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:50.617775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cdf06-2aca-4791-a4df-3e6573aba643 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.618079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cdf06-2aca-4791-a4df-3e6573aba643 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.618343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cdf06-2aca-4791-a4df-3e6573aba643 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.620202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 131/279] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:50.628133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.628461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.628701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.629583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaf86d-bd69-4984-899f-13a34839de33 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:50.654623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:50.656768 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:50.680047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:50.680733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.681017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.681329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.682194 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, user.id=208021a9d5f74cc2813f19291070a807, scope.system=all) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:50.682508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:50.684011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:50.690491 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 149/280] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 15:01:50 2022] GET /identity/v3/role_assignments?user.id=208021a9d5f74cc2813f19291070a807&scope.system=all&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:50.753128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 132/281] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:50.781752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:50.899274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc50ae-8de4-4651-ab83-db5e1958eee1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:50.935756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:50.936842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:50.937307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:50.937782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:50.948906 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:50.956940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:50.958249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:50.959414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:50.962022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 150/282] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:50.968126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:51.065852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:51.119984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693d839-0040-40e9-8d3d-07c7776748ce None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:51.174287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:51.174977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.175265 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.175546 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.176326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:51.176635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:51.178081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4c1e7-fa74-4ade-aa17-cb46d7581a01 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:51.193338 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 133/283] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:51.199342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:51.221630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3ed24-820f-4492-97ff-0d95b0b10247 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.221928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3ed24-820f-4492-97ff-0d95b0b10247 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.222211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3ed24-820f-4492-97ff-0d95b0b10247 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.227470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 134/284] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:51.228733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.229032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.229339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.239985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:51.262297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:51.263841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:51.348643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66826ee9-052f-456c-b195-0c74885b99af None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:51.420722 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:51.422060 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.423559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 135/285] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:51.424063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.424542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] PATH_INFO: `/v3/users/868d258fe9ec4af7abde27bad45fef91` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.430197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] RBAC: Authorizing `identity:get_user(user_id=868d258fe9ec4af7abde27bad45fef91)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:51.431925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:51.434157 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 151/286] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:51 2022] GET /identity/v3/users/868d258fe9ec4af7abde27bad45fef91 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:51.439628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:51.480530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:51.498933 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:51.546552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a1494-bcf6-48c1-8c6f-ad5cf95b2395 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:51.582751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:51.583985 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.584561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.585026 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.586479 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:51.588198 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:51.590634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 136/287] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:51 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:51.596801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:51.696619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:51.721457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76855064-d603-4a01-a7da-93a8c9dba8dc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:51.785444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be82af5-c5db-4dc6-9b93-b7e023f12231 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:51.789156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:51.790287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.790805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.791285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.814835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:51.830379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:51.831531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:51.832479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:51.835812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 152/288] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:51.841613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:51.846275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:51.855112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:51.856169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:51.856538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:51.856888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/868d258fe9ec4af7abde27bad45fef91/roles/f89d9bb93fd142f4bb72e322d8502a85` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:51.859524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user_id=868d258fe9ec4af7abde27bad45fef91, role_id=f89d9bb93fd142f4bb72e322d8502a85)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:51.860780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:51.882400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 137/289] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:01:51 2022] PUT /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/users/868d258fe9ec4af7abde27bad45fef91/roles/f89d9bb93fd142f4bb72e322d8502a85 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:52.016023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3d2be-88e4-4969-93bc-c30c4ab840ac None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:52.064872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:52.065608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:52.065899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:52.066179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:52.067035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:52.067350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:52.068949 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:52.081686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 153/290] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:52.087597 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:52.199083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb174b68-5806-4b32-8bcc-120e7b21f3f4 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:52.235139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:52.236032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:52.236319 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:52.236598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] PATH_INFO: `/v3/users/61218c92925543e8b9ae7dcee110e605` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:52.237854 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] RBAC: Authorizing `identity:get_user(user_id=61218c92925543e8b9ae7dcee110e605)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:52.239344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:52.245384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 138/291] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:52 2022] GET /identity/v3/users/61218c92925543e8b9ae7dcee110e605 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:52.247152 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:52.260673 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:52.400394 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0322ccf-5755-45ca-93c0-519bb279d1d1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:52.440757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:52.441824 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:52.442355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:52.442857 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:52.531357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:52.533311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:52.545302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:52.665620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 154/292] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:52.671097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:52.746882 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:52.808259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceb385-9ce9-479a-bf80-f66bbce42895 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:52.848899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb2791-f1b2-4818-9b5d-4b1d264e49ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:52.849822 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2791-f1b2-4818-9b5d-4b1d264e49ff None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:52.850215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2791-f1b2-4818-9b5d-4b1d264e49ff None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:52.850503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2791-f1b2-4818-9b5d-4b1d264e49ff None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:52.851832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb2791-f1b2-4818-9b5d-4b1d264e49ff None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:52.853462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb2791-f1b2-4818-9b5d-4b1d264e49ff None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:52.855226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 139/293] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:52 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:52.861462 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:52.980697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797b2b4-22bf-4304-ba49-d4e98e4f292b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:52.992440 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:53.020860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:53.021942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.022483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.022990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.024048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=61218c92925543e8b9ae7dcee110e605) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:53.024571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:53.026354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6d96-8729-45d7-a21b-d7524106a05c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:53.033511 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 155/294] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:52 2022] GET /identity/v3/role_assignments?user.id=61218c92925543e8b9ae7dcee110e605&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:53.120910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f365-2a8c-41b2-b74b-dcac3c93ebf8 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.121295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f365-2a8c-41b2-b74b-dcac3c93ebf8 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.121504 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f365-2a8c-41b2-b74b-dcac3c93ebf8 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.123238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 140/295] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:53.129161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9837f93-4d9d-4ba8-a2d6-c69a13b62897 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.129488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9837f93-4d9d-4ba8-a2d6-c69a13b62897 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.129767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9837f93-4d9d-4ba8-a2d6-c69a13b62897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.149291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-a9837f93-4d9d-4ba8-a2d6-c69a13b62897 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:53.150657 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-a9837f93-4d9d-4ba8-a2d6-c69a13b62897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:53.247792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 156/296] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:53.290409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f365-2a8c-41b2-b74b-dcac3c93ebf8 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:53.340904 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af9466-0040-4f53-92cb-02f9e0acf84e None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.341258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af9466-0040-4f53-92cb-02f9e0acf84e None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.341497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af9466-0040-4f53-92cb-02f9e0acf84e None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.343175 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 157/297] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:53.348612 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.348915 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.349193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.383516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:53.384644 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:53.426144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f365-2a8c-41b2-b74b-dcac3c93ebf8 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:53.451088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:53.474562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64538b-8a96-4942-85fe-7fd2dbd39312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:53.474562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64538b-8a96-4942-85fe-7fd2dbd39312 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.474883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64538b-8a96-4942-85fe-7fd2dbd39312 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.475139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64538b-8a96-4942-85fe-7fd2dbd39312 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.479939 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64538b-8a96-4942-85fe-7fd2dbd39312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:53.479939 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64538b-8a96-4942-85fe-7fd2dbd39312 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:53.480248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 141/298] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:53.481504 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0b46c-cde5-4080-a886-98941c1689ac None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.481792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0b46c-cde5-4080-a886-98941c1689ac None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.482068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0b46c-cde5-4080-a886-98941c1689ac None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.483313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 142/299] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:53.489745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.490158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.490537 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.510121 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 158/300] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:53.510985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:53.511311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:53.512523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:53.557739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:53.603599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 143/301] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:53.605025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:53.643166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0fe9f-ef91-41ec-aa6d-46eb26c18ff6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:53.678291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:53.678974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.679254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.679716 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] PATH_INFO: `/v3/groups` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.680463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:53.682002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:53.705270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 159/302] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 15:01:53 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:53.705270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:53.731207 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f186946-e16c-4e21-8465-33d7bb37c16c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:53.756831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:53.779747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:53.780615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.780901 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.781460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.800131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:53.808400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:53.810001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:53.810960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:53.815090 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 144/303] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:53.816152 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:53.836011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:53.852460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e24a5-f7f3-4793-b778-48a17f4ce6b5 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:53.897707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:53.899848 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.900622 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.901135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.916899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:53.927283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:53.928438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:53.929952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:53.930698 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 160/304] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:53.936606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:53.952400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29bb24-fd7d-4c54-850f-63876c242849 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:53.994956 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:53.995823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:53.995951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:53.996151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:53.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:53.997331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:53.998715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:54.008163 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 145/305] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:54.014112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:54.061503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8f99c-7990-4b07-9947-bb0357f3fab4 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:54.071850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:54.099476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:54.100109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.100396 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.100674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.101475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:54.101787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:54.103315 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:54.117434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 161/306] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:54.118438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:54.147924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:54.153343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953a1c5-3bd8-4d55-bbd5-b99dfb856d96 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:54.194648 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:54.195624 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.195912 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.196193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] PATH_INFO: `/v3/users/208021a9d5f74cc2813f19291070a807` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.197516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] RBAC: Authorizing `identity:get_user(user_id=208021a9d5f74cc2813f19291070a807)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:54.198995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:54.200996 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 146/307] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:54 2022] GET /identity/v3/users/208021a9d5f74cc2813f19291070a807 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:54.206539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:54.256508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27591d3-513f-47e4-a050-c7d61f894056 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:54.304494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:54.305792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.306289 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.306748 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] PATH_INFO: `/v3/users/19992db7e7cc4adf9fe80e895c77420d` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.315112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89d94e-9ae1-4126-a998-0833ae1a9894 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:54.323077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:54.323761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] RBAC: Authorizing `identity:get_user(user_id=19992db7e7cc4adf9fe80e895c77420d)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:54.325348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:54.332045 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 162/308] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:54 2022] GET /identity/v3/users/19992db7e7cc4adf9fe80e895c77420d => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:54.332909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:54.351238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a331e-6902-40a8-8831-6c9bf338eb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:54.352182 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a331e-6902-40a8-8831-6c9bf338eb4e None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.352472 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a331e-6902-40a8-8831-6c9bf338eb4e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.352757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a331e-6902-40a8-8831-6c9bf338eb4e None admin] PATH_INFO: `/v3/system/users/208021a9d5f74cc2813f19291070a807/roles/6cb27cd3219a4df396848116d7e9df7e` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.354386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a331e-6902-40a8-8831-6c9bf338eb4e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=208021a9d5f74cc2813f19291070a807, role_id=6cb27cd3219a4df396848116d7e9df7e)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:54.356021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a331e-6902-40a8-8831-6c9bf338eb4e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:54.369045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 147/309] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 15:01:54 2022] PUT /identity/v3/system/users/208021a9d5f74cc2813f19291070a807/roles/6cb27cd3219a4df396848116d7e9df7e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:01:54.417163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:54.445584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe881e0b-18c3-4ef7-bb2f-cc1b352487ee None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:54.486746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:54.487492 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.487738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.488019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.488923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] RBAC: Adding query filter params (role.id=f89d9bb93fd142f4bb72e322d8502a85, user.id=19992db7e7cc4adf9fe80e895c77420d, scope.system=all) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:54.489259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:54.490842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09caf7-9dc3-4be4-85b3-94237355812b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:54.499151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 163/310] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 15:01:54 2022] GET /identity/v3/role_assignments?user.id=19992db7e7cc4adf9fe80e895c77420d&scope.system=all&role.id=f89d9bb93fd142f4bb72e322d8502a85 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:54.665870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93f75-87b3-4b3d-89e9-08685035afc0 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.666557 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93f75-87b3-4b3d-89e9-08685035afc0 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.667049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93f75-87b3-4b3d-89e9-08685035afc0 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.669217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 148/311] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:54.675515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.676162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.676649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.697589 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:54.699146 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:54.770695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 164/312] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:54.794472 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93f75-87b3-4b3d-89e9-08685035afc0 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:54.903835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93f75-87b3-4b3d-89e9-08685035afc0 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:54.939156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:54.940294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:54.940838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:54.941366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:54.953156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:54.961733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:54.962919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:54.963902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:54.966573 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 149/313] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:54.972777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:55.042101 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:55.078166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56d1e6-076f-4edd-84ff-06a3a915ac0d None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:55.117702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:55.118389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:55.118673 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:55.118952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:55.119715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:55.120028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:55.121591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:55.130764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 165/314] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:55.138570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:55.173810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:55.247673 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460893d8-a70d-40f5-8e27-32c1588afb15 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:55.283083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:55.283962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:55.284247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:55.284526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] PATH_INFO: `/v3/users/868d258fe9ec4af7abde27bad45fef91` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:55.285804 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] RBAC: Authorizing `identity:get_user(user_id=868d258fe9ec4af7abde27bad45fef91)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:55.287270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:55.289353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 150/315] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:55 2022] GET /identity/v3/users/868d258fe9ec4af7abde27bad45fef91 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:55.294794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:55.400300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5550818-2aac-402a-80d7-1c2a71968663 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:55.429410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:55.435707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:55.436397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:55.436686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:55.436967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:55.505872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:55.507330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:55.611727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 166/316] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:01:55.617107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:55.725544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:55.729298 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af902096-e4fa-429a-9ed9-0508646fcaaa None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:55.771670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a412bd0-4425-461b-b907-5c7e6d918f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:55.772830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412bd0-4425-461b-b907-5c7e6d918f11 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:55.773318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412bd0-4425-461b-b907-5c7e6d918f11 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:55.773774 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412bd0-4425-461b-b907-5c7e6d918f11 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:55.775148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a412bd0-4425-461b-b907-5c7e6d918f11 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:55.776853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a412bd0-4425-461b-b907-5c7e6d918f11 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:55.779116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 151/317] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:01:55 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:55.785408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:55.816725 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:55.892959 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12c47-962e-425b-8f08-e0c8e41c3923 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:55.931752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:55.932440 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:55.932727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:55.933007 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:55.933909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] RBAC: Adding query filter params (role.id=f89d9bb93fd142f4bb72e322d8502a85, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, user.id=868d258fe9ec4af7abde27bad45fef91) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:55.934225 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:55.935723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca722860-c18e-4395-81de-6559465e0b7f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:55.943415 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 167/318] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:01:55 2022] GET /identity/v3/role_assignments?user.id=868d258fe9ec4af7abde27bad45fef91&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=f89d9bb93fd142f4bb72e322d8502a85 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:56.100863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b6f69-8be2-45fa-965c-08dc12c1d04f None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.101414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b6f69-8be2-45fa-965c-08dc12c1d04f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.101524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b6f69-8be2-45fa-965c-08dc12c1d04f None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.103194 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 152/319] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:56.109005 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.109791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.109791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.129581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:56.131178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:56.203395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 168/320] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:56.226615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b6f69-8be2-45fa-965c-08dc12c1d04f None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:56.342711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b6f69-8be2-45fa-965c-08dc12c1d04f None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:56.378818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3580553-8fad-4805-8342-f635fb241331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:56.379589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580553-8fad-4805-8342-f635fb241331 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.379874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580553-8fad-4805-8342-f635fb241331 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.380150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580553-8fad-4805-8342-f635fb241331 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.381425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3580553-8fad-4805-8342-f635fb241331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:56.382881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3580553-8fad-4805-8342-f635fb241331 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:56.384745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 153/321] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:01:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:56.390374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:56.416183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:56.493899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2056250-ac5c-413a-8028-e0c63997e893 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:56.528524 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad6434-161f-45a6-bcdf-890bef90568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:56.529198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad6434-161f-45a6-bcdf-890bef90568e None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.529513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad6434-161f-45a6-bcdf-890bef90568e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.529785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad6434-161f-45a6-bcdf-890bef90568e None admin] PATH_INFO: `/v3/groups` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.530502 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad6434-161f-45a6-bcdf-890bef90568e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:56.532051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad6434-161f-45a6-bcdf-890bef90568e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:56.547926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 169/322] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 19 15:01:56 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:01:56.745864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62078c-b222-4c8b-852b-b0f42341f8ca None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.746226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62078c-b222-4c8b-852b-b0f42341f8ca None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.746411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62078c-b222-4c8b-852b-b0f42341f8ca None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.747777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 154/323] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:56.753705 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44016624-6a8d-4b2e-b15b-75ebafa20744 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.753996 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44016624-6a8d-4b2e-b15b-75ebafa20744 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.754279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44016624-6a8d-4b2e-b15b-75ebafa20744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.772129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-44016624-6a8d-4b2e-b15b-75ebafa20744 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:56.773381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-44016624-6a8d-4b2e-b15b-75ebafa20744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:56.848880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 170/324] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:56.853543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0720-9c49-4d5d-8417-0621ea4df155 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.853913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0720-9c49-4d5d-8417-0621ea4df155 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.854270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b0720-9c49-4d5d-8417-0621ea4df155 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.855706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 155/325] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:56.861157 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:56.861474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:56.861756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:56.876084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0720-9c49-4d5d-8417-0621ea4df155 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:56.879908 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:56.880914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:56.894968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:56.930137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:56.972910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 171/326] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:01:56.994191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.019563 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b0720-9c49-4d5d-8417-0621ea4df155 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.069020 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.070832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.071334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.071875 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.085968 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:57.096510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:57.098213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:57.099200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:57.102361 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 156/327] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:01:57.108040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.114435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9329-3974-40f6-9bb4-013b45a570ba None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.156778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.158111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.158736 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.159228 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.171306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:57.182419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:57.183488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:01:57.184434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:01:57.188157 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 172/328] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:57.196352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.212426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:57.227346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:57.233849 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55cb2a-01ec-4768-adf6-87ce13fafcb7 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.290025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.291555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.292005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.292528 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.293565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:57.294088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:57.295929 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:57.305551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 157/329] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:57.310636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.335518 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f24834-ca19-4d72-9410-ae7f09d3559e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.384032 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.384771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.385053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.385383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.386232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] RBAC: Adding query filter params (name=reader) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:57.386554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:57.389218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:57.404437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 173/330] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:01:57.411284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.481867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1cd5f-e059-4e88-9b5b-2e21cdb138de None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.523713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:57.555051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.556884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.557178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.557480 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] PATH_INFO: `/v3/users/208021a9d5f74cc2813f19291070a807` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.559529 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] RBAC: Authorizing `identity:get_user(user_id=208021a9d5f74cc2813f19291070a807)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:57.561012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:57.563041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 158/331] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:57 2022] GET /identity/v3/users/208021a9d5f74cc2813f19291070a807 => generated 293 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:57.568642 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.590761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432fe3a-8f1f-4f63-a24e-71c4d92028db None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.603228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:57.638977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.639908 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.640196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.640559 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] PATH_INFO: `/v3/users/19992db7e7cc4adf9fe80e895c77420d` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.646012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] RBAC: Authorizing `identity:get_user(user_id=19992db7e7cc4adf9fe80e895c77420d)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:57.647594 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:57.649659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 174/332] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:01:57 2022] GET /identity/v3/users/19992db7e7cc4adf9fe80e895c77420d => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:57.655295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:57.705025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d320e-7e62-4ccb-9e37-53fe68061713 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.779391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.779979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.780267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.780548 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.781623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] RBAC: Adding query filter params (role.id=6cb27cd3219a4df396848116d7e9df7e, user.id=208021a9d5f74cc2813f19291070a807, scope.system=all) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:01:57.781943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:57.783420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd1035-c502-4661-8677-0ae17fa0c343 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:57.812564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 159/333] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 15:01:57 2022] GET /identity/v3/role_assignments?user.id=208021a9d5f74cc2813f19291070a807&scope.system=all&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 510 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:01:57.833933 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a268b5-7eb4-4b1f-a2d0-40e79b3808b3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:57.880891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddb8ad-1507-447a-af08-32d016b752ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:57.881819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddb8ad-1507-447a-af08-32d016b752ef None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:57.882105 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddb8ad-1507-447a-af08-32d016b752ef None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:57.882386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddb8ad-1507-447a-af08-32d016b752ef None admin] PATH_INFO: `/v3/system/users/19992db7e7cc4adf9fe80e895c77420d/roles/f89d9bb93fd142f4bb72e322d8502a85` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:57.883930 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddb8ad-1507-447a-af08-32d016b752ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=19992db7e7cc4adf9fe80e895c77420d, role_id=f89d9bb93fd142f4bb72e322d8502a85)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:01:57.885377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddb8ad-1507-447a-af08-32d016b752ef None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:01:57.887141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:57.907053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 175/334] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 19 15:01:57 2022] PUT /identity/v3/system/users/19992db7e7cc4adf9fe80e895c77420d/roles/f89d9bb93fd142f4bb72e322d8502a85 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 15:01:59.578581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1a993-f704-4143-a141-5ab7dd349f76 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:59.579115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1a993-f704-4143-a141-5ab7dd349f76 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:59.579115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1a993-f704-4143-a141-5ab7dd349f76 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:59.580714 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 160/335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:59.590001 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:59.590536 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:59.590536 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:59.614287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:59.615514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:01:59.704117 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 176/336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:01:59.743511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1a993-f704-4143-a141-5ab7dd349f76 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:59.839584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:01:59.861200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1a993-f704-4143-a141-5ab7dd349f76 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:01:59.899779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c3a-cdb4-49ae-a201-33b5df650215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:01:59.900593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c3a-cdb4-49ae-a201-33b5df650215 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:59.900891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c3a-cdb4-49ae-a201-33b5df650215 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:59.901174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c3a-cdb4-49ae-a201-33b5df650215 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:59.916291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0c3a-cdb4-49ae-a201-33b5df650215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-1a8c0c3a-cdb4-49ae-a201-33b5df650215 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:01:59.924435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:01:59.925678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:01:59.926700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 161/337] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:01:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:01:59.933941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:01:59.965408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25358288-bb6f-44b7-9ec5-48b60ce33d0b None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:59.965839 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25358288-bb6f-44b7-9ec5-48b60ce33d0b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:59.966128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25358288-bb6f-44b7-9ec5-48b60ce33d0b None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:59.968002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 162/338] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:01:59.974607 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d3f9d-78f2-45a8-b28a-89c8bebc224b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:01:59.974895 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d3f9d-78f2-45a8-b28a-89c8bebc224b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:01:59.975178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d3f9d-78f2-45a8-b28a-89c8bebc224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:01:59.997506 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-599d3f9d-78f2-45a8-b28a-89c8bebc224b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:01:59.999329 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-599d3f9d-78f2-45a8-b28a-89c8bebc224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:00.062702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd4b04-b422-4e8a-921f-9ad2e3e97c9e None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:00.094399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:00.098780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 163/339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:00.100212 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1297ab4-1d1a-4e47-86c5-208544641d0f None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.100542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1297ab4-1d1a-4e47-86c5-208544641d0f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.101123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1297ab4-1d1a-4e47-86c5-208544641d0f None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.106572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 164/340] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:00.110913 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:00.111908 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.113375 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.113892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.114376 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.116161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.117028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.118569 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] RBAC: Adding query filter params (name=member) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:00.120194 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:00.121842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:00.140820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 177/341] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:01:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:00.142364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:00.142725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:00.144015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:00.235123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 165/342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:00.236075 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:00.244619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:00.321595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1e57c-3f84-431e-9073-4fb18c8d2682 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:00.396693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d651-b2fe-4f8e-a5df-d2308c8a3d85 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:00.407014 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:00.408209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.408209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.408405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.435688 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:00.436880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:00.437881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.438381 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.438878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:00.443975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:00.445023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:00.446031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:00.449381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 178/343] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:00.450477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:00.516845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:00.519185 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:00.521337 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:00.569005 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:00.635303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f46dd-0a56-4dc3-993e-827de867d6f0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:00.679342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 166/344] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:00.685522 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:00.717179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:00.718057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.718342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.718665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.745386 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:00.753564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:00.754620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 179/345] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:00.755574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:00.799945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88525e4d-af19-4a48-b9cc-dac25b537955 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:00.802074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:00.846688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:00.847536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:00.847941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:00.848223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:00.849001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:00.849336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:00.853003 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:00.870345 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 167/346] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:02:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:00.871765 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:00.952113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38cfe-9492-42b9-890d-c211d329f3d7 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:00.973305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.034178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.035492 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.035992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.036491 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.038052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.039756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.044334 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 180/347] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:00 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:01.050370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.070390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8eb6-050c-4c64-b3ef-f651da40e3d2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.111610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.112786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.113450 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.114633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.115820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:01.117678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.119493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.139375 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 168/348] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:01.141350 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.191046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.209579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee916a-7b13-4651-807f-1f3a4a917810 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.247828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af9ae9-6b37-442b-9d0e-23bb2df07520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.248528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9ae9-6b37-442b-9d0e-23bb2df07520 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.249044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9ae9-6b37-442b-9d0e-23bb2df07520 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.249044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9ae9-6b37-442b-9d0e-23bb2df07520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.285324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3571-a111-487b-accc-f407e0f2e290 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.304433 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.325871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.327133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.327642 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.327873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af9ae9-6b37-442b-9d0e-23bb2df07520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.328374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.329415 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af9ae9-6b37-442b-9d0e-23bb2df07520 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.335446 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.337323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.339325 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 169/349] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:01.340478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.434054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 181/350] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:01.435566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4f58-5703-4ca1-be61-00ca5353c135 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.435690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4f58-5703-4ca1-be61-00ca5353c135 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.435922 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4f58-5703-4ca1-be61-00ca5353c135 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.437598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 182/351] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:01.438402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4f58-5703-4ca1-be61-00ca5353c135 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.443792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.455992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98044e64-b615-4a1e-97a0-9a56a6fbaf2b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.496250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.497365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.497976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.498475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.531329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.555680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4f58-5703-4ca1-be61-00ca5353c135 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.580691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.582550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.592841 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.593606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.593892 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.594171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.595019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:01.595334 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.596837 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcbea1-9478-41b5-81ec-4635bbd20f8f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.602950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 183/352] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:01.603992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43991e16-72fb-4d1a-a11f-c02326ea0885 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.604281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43991e16-72fb-4d1a-a11f-c02326ea0885 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.604558 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43991e16-72fb-4d1a-a11f-c02326ea0885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.623110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-43991e16-72fb-4d1a-a11f-c02326ea0885 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:01.624376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-43991e16-72fb-4d1a-a11f-c02326ea0885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:01.690043 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.707354 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 170/353] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:01.708112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 184/354] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:01.709218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.712772 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95525bc-5db3-41e9-96d5-b73dec23fdc1 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.713096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95525bc-5db3-41e9-96d5-b73dec23fdc1 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.713447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95525bc-5db3-41e9-96d5-b73dec23fdc1 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.716061 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 185/355] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:01.719127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95525bc-5db3-41e9-96d5-b73dec23fdc1 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.822212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:01.856113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602d508-22ec-4bdd-9351-571c18e0075b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.865092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95525bc-5db3-41e9-96d5-b73dec23fdc1 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:01.899154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323db954-586b-4039-9ecc-84100101b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.900705 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db954-586b-4039-9ecc-84100101b457 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.900998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db954-586b-4039-9ecc-84100101b457 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.901982 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db954-586b-4039-9ecc-84100101b457 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.902838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db954-586b-4039-9ecc-84100101b457 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.904322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db954-586b-4039-9ecc-84100101b457 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.905750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:01.909018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.909200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 171/356] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:01.909349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.909349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.909508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b317-0d46-4abc-93ea-7b12caef28fc None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:01.909508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b317-0d46-4abc-93ea-7b12caef28fc None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:01.909508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b317-0d46-4abc-93ea-7b12caef28fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:01.909723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:01.911399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:01.913464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 186/357] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:01.914629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:01.934293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e275b317-0d46-4abc-93ea-7b12caef28fc None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:01.935490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e275b317-0d46-4abc-93ea-7b12caef28fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:02.007984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:02.027283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 172/358] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:02.028315 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87779e-e462-4ce2-ba3a-f928f6435845 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.028637 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87779e-e462-4ce2-ba3a-f928f6435845 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.029346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87779e-e462-4ce2-ba3a-f928f6435845 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.031312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 173/359] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:02.032262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87779e-e462-4ce2-ba3a-f928f6435845 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.052435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb98a-195c-4130-a2eb-af31afe8af19 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.097474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.098699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.099080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.099457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.112786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:02.117697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:02.126556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:02.127790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:02.128884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:02.131165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 187/360] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:02.132107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.164596 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87779e-e462-4ce2-ba3a-f928f6435845 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.201888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b303-db77-4132-9e2f-83861c2eeeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.202822 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b303-db77-4132-9e2f-83861c2eeeb9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.203166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b303-db77-4132-9e2f-83861c2eeeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.203445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b303-db77-4132-9e2f-83861c2eeeb9 None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.205439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b303-db77-4132-9e2f-83861c2eeeb9 None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:02.207044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b303-db77-4132-9e2f-83861c2eeeb9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:02.208726 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 174/361] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:02.209923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.210211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.210487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.230034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:02.231244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:02.256999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccf138-4792-4468-b864-c9f3d1fbe814 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.299000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:02.317132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.319018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.319523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.319986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.321502 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:02.322461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 175/362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:02.323399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.323614 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:02.327947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 188/363] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:02.328986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.415342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:02.461364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220d36-24af-4e28-9a0c-6e61e8b26a7b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.462572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2af185-b555-48ac-834a-8fafb16fd99b None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.505034 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.506050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.506624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.506624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.507373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:02.507687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:02.509143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:02.514863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.516006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.516444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.516872 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.518919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 189/364] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:02.519764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.530441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:02.538775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:02.539858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 176/365] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:02.541801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.599795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:02.664083 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fcc07-bc03-4f73-99ba-71c1d42607ad None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.677474 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d7aa-2ad6-41b5-a7a9-107b0c784606 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.698547 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:02.705564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.706408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.707017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.707302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.708485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:02.709135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:02.710637 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:02.717215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 190/366] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:02.718047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.720041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.720719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.721001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.721298 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.722174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:02.722677 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:02.724530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:02.731094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 177/367] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:02.732143 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.862106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbd22c-2ebb-4829-874d-d2c3867b1761 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.919618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:02.920847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:02.921401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:02.922209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:02.946814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:02.955168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:02.956327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:02.957338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:02.958830 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 191/368] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:02.961279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da88b8-a3da-42e0-b57a-d80ada5b8e8f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:02.972671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:02.981648 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.035643 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.036321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.036610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.036887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.073765 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.126687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.128371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.130675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e71e0-c67d-460a-b96c-a733393343eb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:03.184732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.185753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.186212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.186658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.187645 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:03.188117 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.189723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.211367 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 192/369] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:03.212612 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:03.235746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 178/370] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:03.236751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:03.257535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.350020 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea2790-903f-4a9c-babc-f83c59214aa3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:03.386709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.387405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.387692 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.387969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.388999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:03.389127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.389629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.392848 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.402404 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 193/371] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:03.405382 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:03.417268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f53f1-14df-4b32-979e-104556b82d7f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:03.510433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.511259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.511541 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.511817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] PATH_INFO: `/v3/users/19992db7e7cc4adf9fe80e895c77420d` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.513754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] RBAC: Authorizing `identity:get_user(user_id=19992db7e7cc4adf9fe80e895c77420d)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.515248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.517572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 179/372] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:02:03 2022] GET /identity/v3/users/19992db7e7cc4adf9fe80e895c77420d => generated 293 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:03.522591 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:03.528341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd489c8-4d38-4aea-b28d-739d0321225b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:03.594544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.595430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.595705 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.595990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.600323 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.601849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.603845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 194/373] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:03 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:03.610406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:03.664906 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.690031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19758334-6281-4d77-a54c-1b449670eca2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:03.722251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.736650 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.737652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.737652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.738185 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.819943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.823575 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.843191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892e480-d7df-43c1-a583-2629ab69fd45 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:03.916114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:03.917223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:03.917741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:03.918229 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:03.919742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] RBAC: Adding query filter params (role.id=f89d9bb93fd142f4bb72e322d8502a85, user.id=19992db7e7cc4adf9fe80e895c77420d, scope.system=all) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:03.920303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:03.923107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:03.963890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:03.964709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 195/374] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 19 15:02:03 2022] GET /identity/v3/role_assignments?user.id=19992db7e7cc4adf9fe80e895c77420d&scope.system=all&role.id=f89d9bb93fd142f4bb72e322d8502a85 => generated 510 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:03.966276 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:04.042230 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 180/375] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:04.051466 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:04.169185 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1b78c-21a5-4b97-b51e-9ef84082156c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:04.174173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:04.259745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:04.261378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.261892 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.262604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.267390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:04.269944 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:04.273220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 196/376] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:03 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:04.291509 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:04.379076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eef096-5c1c-4a5c-a3ce-3d7bb79c4c3c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:04.430636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:04.431443 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.431724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.431993 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] PATH_INFO: `/v3/projects/97613977fc9648acbe5c5524c71c9426` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.438016 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] RBAC: Authorizing `identity:get_project(project_id=97613977fc9648acbe5c5524c71c9426)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:04.439579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:04.441791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 181/377] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:04 2022] GET /identity/v3/projects/97613977fc9648acbe5c5524c71c9426 => generated 344 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:04.449861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:04.494932 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ec7b1-32a4-44e0-87b1-8f86a3cbc36b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:04.505655 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:04.568323 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:04.570128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.570625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.570812 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7224db-f98b-4c23-880c-765e84972480 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:04.571120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.572850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:04.573560 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:04.581058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7498e-9aad-4e76-8d8d-c0d64939b6d3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:04.593445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 197/378] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:04 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 260 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:04.609341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788db0b-793b-47eb-80a8-bfa84c3f3700 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.609950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788db0b-793b-47eb-80a8-bfa84c3f3700 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.610264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:04.610420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788db0b-793b-47eb-80a8-bfa84c3f3700 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.611149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.611430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.611704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] PATH_INFO: `/v3/groups/a4a23188e9104db9bf5975359d8fce81` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.613555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 198/379] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:04.617842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] RBAC: Authorizing `identity:get_group(group_id=a4a23188e9104db9bf5975359d8fce81)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:04.619317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:04.621155 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 182/380] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:04 2022] GET /identity/v3/groups/a4a23188e9104db9bf5975359d8fce81 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:04.622160 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.622649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.623116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.627160 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:04.654625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:04.656789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:04.712663 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:04.738240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa17275-b62b-42d1-aef7-4fa6e4910c83 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:04.749928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:04.775514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:04.776176 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:04.776458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:04.776748 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:04.777611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] RBAC: Adding query filter params (group.id=a4a23188e9104db9bf5975359d8fce81, role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=97613977fc9648acbe5c5524c71c9426) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:04.777919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:04.779388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:04.785082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 199/381] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:04.789626 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 183/382] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:04 2022] GET /identity/v3/role_assignments?group.id=a4a23188e9104db9bf5975359d8fce81&scope.project.id=97613977fc9648acbe5c5524c71c9426&role.id=8f859c3494964044956fd86bc077eaf0 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:04.822580 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:05.008469 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a0b77-864f-4af8-a7d6-538d286581af None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:05.071952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:05.073659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:05.074358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:05.075078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:05.095906 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:05.111747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:05.114046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:05.116179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:05.120699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 200/383] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:05.130283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:05.361940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95c07-c6c6-4028-b6a6-982fc9e4c2f8 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:05.427098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:05.428209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:05.428707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:05.429200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:05.430664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:05.431222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:05.433883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:05.450958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 184/384] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:02:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:05.460056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:05.507777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:05.598193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d55688-8011-4d5a-bd96-8a3d70136afe None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:05.645723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:05.646603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:05.646886 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:05.647162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:05.648358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:05.649886 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:05.652544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 201/385] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:05 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:05.657316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:05.737212 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:05.821193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e722c9-a567-406c-a808-12291cc83502 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:05.863690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b0bbd-c0d2-4e90-832f-f5c255d3dca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:05.864676 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b0bbd-c0d2-4e90-832f-f5c255d3dca9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:05.864969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b0bbd-c0d2-4e90-832f-f5c255d3dca9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:05.865286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b0bbd-c0d2-4e90-832f-f5c255d3dca9 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:05.866496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b0bbd-c0d2-4e90-832f-f5c255d3dca9 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:05.867977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b0bbd-c0d2-4e90-832f-f5c255d3dca9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:05.870091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 185/386] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:05 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:05.875894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:05.983215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:05.989262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3ca20-7dbd-4d94-ac94-9d089186e491 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:06.032751 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af3245-5d32-481d-a011-9c1d7f666c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:06.033700 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3245-5d32-481d-a011-9c1d7f666c79 None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.033996 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3245-5d32-481d-a011-9c1d7f666c79 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.034271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3245-5d32-481d-a011-9c1d7f666c79 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/groups/8c06d1db29434d569b96ef655df43084/roles/6cb27cd3219a4df396848116d7e9df7e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.036403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af3245-5d32-481d-a011-9c1d7f666c79 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac748ce4db4449de9948ac8e1f93cc8b, group_id=8c06d1db29434d569b96ef655df43084, role_id=6cb27cd3219a4df396848116d7e9df7e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:06.037962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af3245-5d32-481d-a011-9c1d7f666c79 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:06.065631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 202/387] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 15:02:05 2022] PUT /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/groups/8c06d1db29434d569b96ef655df43084/roles/6cb27cd3219a4df396848116d7e9df7e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:06.301322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aae001-aa7d-4456-a83d-957f03842af2 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.301703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aae001-aa7d-4456-a83d-957f03842af2 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.301963 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aae001-aa7d-4456-a83d-957f03842af2 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.303497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 186/388] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:06.310030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb2e5f-ecda-42a2-b517-5bb2f5fb167f None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.310361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb2e5f-ecda-42a2-b517-5bb2f5fb167f None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.310602 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb2e5f-ecda-42a2-b517-5bb2f5fb167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.332789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-b6eb2e5f-ecda-42a2-b517-5bb2f5fb167f None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:06.334145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-b6eb2e5f-ecda-42a2-b517-5bb2f5fb167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:06.417366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 203/389] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:06.437694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aae001-aa7d-4456-a83d-957f03842af2 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:06.514517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75739be-3220-4fdb-8869-68285c07fda0 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.515263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75739be-3220-4fdb-8869-68285c07fda0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.515754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75739be-3220-4fdb-8869-68285c07fda0 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.518831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 204/390] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:06.524783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.525510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.526010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.546964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:06.549325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:06.557263 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aae001-aa7d-4456-a83d-957f03842af2 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:06.560007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:06.593913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:06.594798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.595091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.595378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.606971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:06.618609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:06.619772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:06.620734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 187/391] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:06.626076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:06.669754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 205/392] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:06.704608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:06.751355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:06.769263 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1f20c-49ba-471e-8ce7-813272fe21cd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:06.812304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:06.812898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:06.813274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:06.813492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:06.814267 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:06.814579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:06.818609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:06.844029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 188/393] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:06.851990 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:06.903657 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:06.974733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8634e-e20f-42f1-b2fb-ea5f7b61bfa6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:06.992644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab561-0674-4b48-8bcd-c2e3fc2be2ef None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:07.029616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.030651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.031021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.031424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.032772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:07.034444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:07.036772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 189/394] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:06 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:07.042718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:07.114713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.115667 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.116885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.116885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.145738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:07.153998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899139e-fdae-4f3a-a6f4-329ac03b2334 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:07.155992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:07.170931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:07.172069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:07.173009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 206/395] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:07.178119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:07.189623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.191388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.191680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.191966 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.193214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:07.194736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:07.197322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 190/396] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:07 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:07.203040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:07.349678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157b40-afde-4e9e-acae-bc708945b10a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:07.386597 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.387583 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.388715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.389014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/groups/8c06d1db29434d569b96ef655df43084/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.391442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] RBAC: Authorizing `identity:create_grant(project_id=ac748ce4db4449de9948ac8e1f93cc8b, group_id=8c06d1db29434d569b96ef655df43084, role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:07.393522 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:07.397634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:07.430937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09789be5-a415-4703-9789-ec2d6c7b718e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:07.435557 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 191/397] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 15:02:07 2022] PUT /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b/groups/8c06d1db29434d569b96ef655df43084/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:07.479894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.480621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.480904 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.481179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.482023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] RBAC: Adding query filter params (name=member) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:07.482334 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:07.484015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145a88d-7a66-4a96-a099-5fb11f1fd59f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:07.502772 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 207/398] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:02:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:07.511385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:07.556589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:07.673263 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0124f-84a8-4336-9491-ed3d079cc56e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:07.688283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0025d-2389-4dc9-b4f5-318b594c2cdd None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.688657 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0025d-2389-4dc9-b4f5-318b594c2cdd None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.688857 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0025d-2389-4dc9-b4f5-318b594c2cdd None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.691815 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 208/399] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:07.696366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc99ed-3886-47e0-b5a9-78b38316334a None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.696664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc99ed-3886-47e0-b5a9-78b38316334a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.696937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc99ed-3886-47e0-b5a9-78b38316334a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.720524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.721940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.722422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.722906 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.724435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:07.726183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:07.728535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 192/400] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:07 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:07.732263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-46dc99ed-3886-47e0-b5a9-78b38316334a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:07.733537 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-46dc99ed-3886-47e0-b5a9-78b38316334a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:07.734305 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:07.831112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 209/401] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:07.850466 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e36df-2aa6-4c51-bb26-aed920392ba5 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.850844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e36df-2aa6-4c51-bb26-aed920392ba5 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.851064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e36df-2aa6-4c51-bb26-aed920392ba5 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.852719 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 210/402] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:07.854953 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:07.859112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.859406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.859687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.868916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdebad-b833-4eeb-8e7c-6d287cb73883 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:07.877770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:07.878985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:07.907558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:07.908867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:07.909447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:07.909919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:07.911454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:07.913296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:07.915750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 193/403] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:07 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:07.916689 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:07.953114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:07.963260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 211/404] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:07.964336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.040132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e65daf-682f-4926-9061-9dc2218065d2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.080178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.081581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.082205 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.082855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.094114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909159b6-381c-421c-b040-7fb4695d5f32 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.098671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:08.106934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:08.108168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:08.109508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:08.117942 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 194/405] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:08.119179 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.144036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.145375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.145866 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.146316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/groups/8c06d1db29434d569b96ef655df43084/roles/6cb27cd3219a4df396848116d7e9df7e` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.149003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] RBAC: Authorizing `identity:create_grant(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, group_id=8c06d1db29434d569b96ef655df43084, role_id=6cb27cd3219a4df396848116d7e9df7e)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:08.150786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:08.157961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:08.178036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 212/406] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 15:02:07 2022] PUT /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/groups/8c06d1db29434d569b96ef655df43084/roles/6cb27cd3219a4df396848116d7e9df7e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:08.179168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.276029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:08.324266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3ec94-08bd-43db-a4df-84bcc82d83b0 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.389250 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94865b-adf3-438d-a8e3-8657cfe59a2d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.396743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.397736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.398123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.398506 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.410988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:08.419776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:08.421817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:08.423406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:08.425551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 195/407] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:08.428387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.435503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.436452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.436914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.437576 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.438543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:08.439263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:08.440856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:08.450604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 213/408] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:08.464213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.519691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:08.543951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce20302-d262-4fe0-a1ec-4fa6571d5e38 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.590139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.591189 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.591744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.592305 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.593377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:08.593889 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:08.595608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:08.613086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 196/409] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:08.631010 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.662626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61902dc8-3ebb-4bcd-a135-98ea0d968a00 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.685608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:08.701701 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.702718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.703077 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.703428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] PATH_INFO: `/v3/groups/a4a23188e9104db9bf5975359d8fce81` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.704755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] RBAC: Authorizing `identity:get_group(group_id=a4a23188e9104db9bf5975359d8fce81)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:08.706579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:08.713253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 214/410] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:08 2022] GET /identity/v3/groups/a4a23188e9104db9bf5975359d8fce81 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:08.714789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.829629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541e952-53e2-43b7-9dcf-10257ecdf50a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.854594 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6207d89-0f46-4bae-9c82-305f2af0eaf6 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:08.873265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.873265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.873265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.873265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] PATH_INFO: `/v3/projects/97613977fc9648acbe5c5524c71c9426` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.873726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] RBAC: Authorizing `identity:get_project(project_id=97613977fc9648acbe5c5524c71c9426)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:08.873726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:08.874205 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 215/411] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:08 2022] GET /identity/v3/projects/97613977fc9648acbe5c5524c71c9426 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:08.880911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:08.953708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:08.985539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:08.986441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:08.986742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:08.987026 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:08.988241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:08.989883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:08.996394 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 197/412] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:08 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:09.002028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46d9d7-3c35-4e2f-bc49-3bf48d6c8b73 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:09.006647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:09.035347 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:09.049222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d395f5-e0de-4e4b-8c59-65e2dd54b40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:09.050544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d395f5-e0de-4e4b-8c59-65e2dd54b40a None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.052170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d395f5-e0de-4e4b-8c59-65e2dd54b40a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.052680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d395f5-e0de-4e4b-8c59-65e2dd54b40a None admin] PATH_INFO: `/v3/projects/97613977fc9648acbe5c5524c71c9426/groups/a4a23188e9104db9bf5975359d8fce81/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.058120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d395f5-e0de-4e4b-8c59-65e2dd54b40a None admin] RBAC: Authorizing `identity:create_grant(project_id=97613977fc9648acbe5c5524c71c9426, group_id=a4a23188e9104db9bf5975359d8fce81, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:09.061357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d395f5-e0de-4e4b-8c59-65e2dd54b40a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:09.096988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 216/413] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 15:02:08 2022] PUT /identity/v3/projects/97613977fc9648acbe5c5524c71c9426/groups/a4a23188e9104db9bf5975359d8fce81/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:09.167708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f5419-d345-401e-9867-f6a4846b4d37 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.168368 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f5419-d345-401e-9867-f6a4846b4d37 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.168895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f5419-d345-401e-9867-f6a4846b4d37 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.170663 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 217/414] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:09.183479 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e7c9a-9f6a-4e14-9b9d-641f6f653bdc None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:09.187323 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.187894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.188397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.219095 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:09.221610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:09.266403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:09.268177 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.268720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.269218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.271107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:09.272901 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:09.281016 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 198/415] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:09 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:09.283088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:09.373617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:09.389881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 218/416] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:09.401099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa3d6-3344-4fd2-95f2-a7bdc8d3bf67 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:09.415179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:09.441587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:09.442772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.443068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.443352 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/groups/8c06d1db29434d569b96ef655df43084/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.446778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] RBAC: Authorizing `identity:create_grant(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02, group_id=8c06d1db29434d569b96ef655df43084, role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:09.449402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:09.465938 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 199/417] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 19 15:02:09 2022] PUT /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02/groups/8c06d1db29434d569b96ef655df43084/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 15:02:09.491862 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:09.553778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aff8f4-79dd-49b7-bea3-a12fa7b68bea None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:09.598162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:09.599403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.599881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.600356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] PATH_INFO: `/v3/roles/member` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.612789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:09.621219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:09.622384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:09.623348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:09.626415 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 219/418] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:09.632040 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:09.777778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6216c0f-9fc1-4527-a6fe-159dfc8151aa None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:09.813911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:09.814661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:09.814951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:09.815282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:09.816052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:09.816410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:09.817828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:09.820181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:09.828162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 200/419] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:02:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:09.834887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:09.978585 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930a7c4-51f4-4900-a4a0-b511edc2affb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:09.980829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:10.030103 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5514f9e-3a10-4e74-a564-d637df9034de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:10.030865 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5514f9e-3a10-4e74-a564-d637df9034de None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.031153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5514f9e-3a10-4e74-a564-d637df9034de None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.031431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5514f9e-3a10-4e74-a564-d637df9034de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.120010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5514f9e-3a10-4e74-a564-d637df9034de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:10.121524 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5514f9e-3a10-4e74-a564-d637df9034de None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:10.221298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 220/420] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:10.227297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:10.265448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b16d92-31af-4107-b21b-4aa51a59cd73 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.265448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b16d92-31af-4107-b21b-4aa51a59cd73 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.265748 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b16d92-31af-4107-b21b-4aa51a59cd73 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.269321 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 221/421] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:10.273608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.273971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.274322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.285390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:10.299764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:10.301840 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:10.377558 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 222/422] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:10.424824 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:10.452121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe7516-61ec-4b5d-b43f-23f941481b26 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:10.527979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:10.528787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.529060 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.529366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.534018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:10.535457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:10.541265 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 201/423] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:10 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:10.555218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:10.561937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e5e8-b798-4b1b-bbd1-25a1644fae54 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:10.599945 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:10.601148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.601708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.602170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.604385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:10.615937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:10.624555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:10.625601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:10.626666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:10.629339 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 223/424] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:10.634960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:10.716823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:10.748228 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d08af9-40ad-44d5-ad98-2b0658898ab6 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:10.787774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:10.788746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.789205 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.789714 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.790722 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:10.791232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:10.792834 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:10.802708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 224/425] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:10.806073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0fb11-02d5-4463-9843-a01810bf1e10 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:10.808897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:10.884485 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:10.885407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.885699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.885978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:10.889975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:10.891605 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:10.894029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 202/426] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:10 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:10.897604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:10.901596 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:10.948248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831c22c-3ead-4487-9fc1-b7cd4590794d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:10.983157 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:10.983818 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:10.984118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:10.989262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.057209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:11.059005 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:11.077309 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c58c8-d868-40a0-91ff-6f7dc874541b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.110908 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.111568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.111899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.112178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.113065 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:11.113414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:11.115023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864acff-c0e1-4454-9569-f31263a3820c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:11.117795 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:11.121654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:11.125691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 203/427] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:10 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 611 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:11.127275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a35d1a-1d41-41b7-9d60-94200f8a160b None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.127631 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a35d1a-1d41-41b7-9d60-94200f8a160b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.127983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a35d1a-1d41-41b7-9d60-94200f8a160b None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.133810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 204/428] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:11.135535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.135919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.136263 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.176960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 225/429] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:11.178654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:11.179947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:11.182431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:11.301120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c701-347a-4b28-8420-ea3bdf46a9dd None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.341303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.342534 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.343008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.343520 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] PATH_INFO: `/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.345056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] RBAC: Authorizing `identity:get_project(project_id=ac748ce4db4449de9948ac8e1f93cc8b)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:11.346799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:11.349172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 226/430] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:11 2022] GET /identity/v3/projects/ac748ce4db4449de9948ac8e1f93cc8b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:11.355994 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:11.368840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 205/431] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:11.388749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:11.413355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:11.479798 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef308a-f0b9-4f29-91ad-17a4a353f50b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.520759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990744f8-b190-4525-985f-64851f02f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.521664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990744f8-b190-4525-985f-64851f02f52f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.521945 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990744f8-b190-4525-985f-64851f02f52f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.522217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990744f8-b190-4525-985f-64851f02f52f None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.523422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990744f8-b190-4525-985f-64851f02f52f None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:11.524986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990744f8-b190-4525-985f-64851f02f52f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:11.526970 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 227/432] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:11 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:11.534224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990744f8-b190-4525-985f-64851f02f52f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:11.540508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c2ff-bb24-4af9-93f1-fff48631d800 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.573648 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:11.577952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.578958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.579340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.579715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.591610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:11.599351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:11.600574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 15:02:11.601564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:11.602284 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 206/433] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:11.607894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:11.651264 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:11.667975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990744f8-b190-4525-985f-64851f02f52f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.708679 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.709801 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.710304 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.710818 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.711877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:11.712405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:11.714120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:11.721845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 228/434] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:11 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=ac748ce4db4449de9948ac8e1f93cc8b&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:11.737799 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8a5b9-79a0-4198-b784-5a9016654702 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.778727 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.779840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.780341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.780932 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:11.781981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] RBAC: Adding query filter params (name=member) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:11.782514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:11.784174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875e9c-3614-4b60-b08a-cd98e2e4a02e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:11.794512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 207/435] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:02:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:11.802458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:11.926893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5b367-f2e4-4c94-9313-8fbb334f21fb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:11.964660 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45de44-3efe-4ca5-8c55-e9999aa15dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:11.965837 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45de44-3efe-4ca5-8c55-e9999aa15dc7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:11.966349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45de44-3efe-4ca5-8c55-e9999aa15dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:11.966875 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45de44-3efe-4ca5-8c55-e9999aa15dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.000402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.051236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45de44-3efe-4ca5-8c55-e9999aa15dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:12.052724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45de44-3efe-4ca5-8c55-e9999aa15dc7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:12.060102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9889f6-e1be-40b4-aa12-178770486682 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.060430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9889f6-e1be-40b4-aa12-178770486682 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.060676 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9889f6-e1be-40b4-aa12-178770486682 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.062370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 208/436] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:12.070261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05697485-ca36-4a7d-82fd-1f9be39313c8 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.070593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05697485-ca36-4a7d-82fd-1f9be39313c8 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.070824 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05697485-ca36-4a7d-82fd-1f9be39313c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.096539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-05697485-ca36-4a7d-82fd-1f9be39313c8 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:12.098391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-05697485-ca36-4a7d-82fd-1f9be39313c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:12.163922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.170543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 229/437] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:12.176696 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bcc68-793b-46cf-82bb-7cf9be4261a6 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.177362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bcc68-793b-46cf-82bb-7cf9be4261a6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.177901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bcc68-793b-46cf-82bb-7cf9be4261a6 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.179494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 230/438] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:12.180469 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bcc68-793b-46cf-82bb-7cf9be4261a6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.192074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 209/439] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:12.193865 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.194467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.194950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.214467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:12.215641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:12.243881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.298847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bcc68-793b-46cf-82bb-7cf9be4261a6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.305194 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 210/440] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:12.305863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.336986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.338059 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.338343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.338620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.340331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:12.341872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:12.343743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 231/441] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:12.344608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.413547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949efd-4025-4677-adad-8b5585bc0232 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.423995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.444178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1da16-be36-4a9c-b3e5-e4d22fbe092e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.447486 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.448293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.448579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.448857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.459093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:12.466704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:12.467946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 211/442] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:12.468979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.470520 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.483348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.484196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.484482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.484760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.495020 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:12.503224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:12.504486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 232/443] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:12.505918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.576168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a8c7-a5be-4ee8-adc0-e39524cf0958 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.604936 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c36b7-94b5-40ac-953d-a9a46997276c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.608976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.609878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.610161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.610434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.611618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:12.613092 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:12.614769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 212/444] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:12.615519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.638924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.639599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.639873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.640209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.640970 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:12.641307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:12.643094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:12.655722 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 233/445] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:12.655722 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.659327 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.699118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.720553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab9581-2f07-496c-9411-7a7df5d3c8f4 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.752903 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742d445-6fb0-4902-ba55-82412721d0fc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.753814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.754490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.754773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.755061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.755809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:12.756121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:12.757521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:12.765487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 213/446] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:12.766275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.790598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.791303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.791576 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.791855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.792695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=6cb27cd3219a4df396848116d7e9df7e, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:12.793010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:12.794508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:12.800348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 234/447] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=6cb27cd3219a4df396848116d7e9df7e => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:12.801127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:12.879609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.883214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62042e-f1ba-4f2d-b29b-3b3e1ca5b940 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.919474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a425-ca4e-4d05-ac0f-9b7adddc469c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:12.924534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.925204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.925517 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.925794 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:12.932387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:12.960245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:12.960851 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:12.961133 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:12.961437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.007960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.009720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.037503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.038974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.119953 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:13.124538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 214/448] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:13.129471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:13.159758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 235/449] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:13.165519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:13.179182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:13.243247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958ff9b-3d49-47e7-b5cf-a053a711f65d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:13.274884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e15ade-524d-4b6d-903f-7081f99625a1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:13.278415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:13.279288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:13.279572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:13.279851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] PATH_INFO: `/v3/projects/97613977fc9648acbe5c5524c71c9426` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.281109 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=97613977fc9648acbe5c5524c71c9426)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.282593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.284326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 215/450] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:13 2022] GET /identity/v3/projects/97613977fc9648acbe5c5524c71c9426 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:13.289915 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:13.308646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:13.309658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:13.309970 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:13.310247 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] PATH_INFO: `/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.311794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] RBAC: Authorizing `identity:get_project(project_id=9a15fd6fd3ce4d6a857a55cc8f2b5f02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.313342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.315045 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 236/451] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:02:13 2022] GET /identity/v3/projects/9a15fd6fd3ce4d6a857a55cc8f2b5f02 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:13.320375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:13.355334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:13.390215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a726da-ff1e-480e-9428-5fed0e0c2cf5 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:13.396542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:13.417901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752da43e-a7d4-431b-92d7-7f26781ffecf None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:13.423486 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:13.424342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:13.424620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:13.424893 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] PATH_INFO: `/v3/groups/a4a23188e9104db9bf5975359d8fce81` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.426097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] RBAC: Authorizing `identity:get_group(group_id=a4a23188e9104db9bf5975359d8fce81)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.427532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.429155 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 216/452] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:13 2022] GET /identity/v3/groups/a4a23188e9104db9bf5975359d8fce81 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:13.434944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:13.452356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:13.453297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:13.453599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:13.453849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] PATH_INFO: `/v3/groups/8c06d1db29434d569b96ef655df43084` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.455064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] RBAC: Authorizing `identity:get_group(group_id=8c06d1db29434d569b96ef655df43084)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.456505 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.458294 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 237/453] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 19 15:02:13 2022] GET /identity/v3/groups/8c06d1db29434d569b96ef655df43084 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:13.464176 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:13.532613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6db46-7cb9-49de-96c5-878994b6ea18 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:13.563764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4a293-f9d5-4b32-b96c-dd5d25c7140c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:13.566960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:13.567613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:13.567896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:13.568172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.569001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] RBAC: Adding query filter params (group.id=a4a23188e9104db9bf5975359d8fce81, role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=97613977fc9648acbe5c5524c71c9426) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:13.569341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.571166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02a6f-c4d0-4249-8ad3-73e01d43e1bb None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.574681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:13.581632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 217/454] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:13 2022] GET /identity/v3/role_assignments?group.id=a4a23188e9104db9bf5975359d8fce81&scope.project.id=97613977fc9648acbe5c5524c71c9426&role.id=8f859c3494964044956fd86bc077eaf0 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:13.598935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:13.599627 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:13.599907 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:13.600180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:13.601042 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] RBAC: Adding query filter params (group.id=8c06d1db29434d569b96ef655df43084, role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:13.601407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:13.602911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff064694-ab3c-436b-b4f8-4e4dbf89d083 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:13.609985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 238/455] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 19 15:02:13 2022] GET /identity/v3/role_assignments?group.id=8c06d1db29434d569b96ef655df43084&scope.project.id=9a15fd6fd3ce4d6a857a55cc8f2b5f02&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:16.385121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7ce79-ddf0-4af5-8042-056439f59f3d None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.385716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7ce79-ddf0-4af5-8042-056439f59f3d None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.385716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7ce79-ddf0-4af5-8042-056439f59f3d None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.387407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 218/456] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:16.393053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.393933 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.394428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.425401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:16.426981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:16.445285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:16.548756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 239/457] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:16.585769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7ce79-ddf0-4af5-8042-056439f59f3d None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:16.706590 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7ce79-ddf0-4af5-8042-056439f59f3d None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:16.743005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:16.744045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.744331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.744605 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.756032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:16.757759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c8a16996-2b93-469f-8247-f887f0a5248e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:16.778064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:16.779159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:16.780172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:16.780172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:16.780172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:16.780172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:16.780172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 219/458] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:16.787519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:16.851454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a55d3-82d3-4615-a418-e9000949275e None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.851454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a55d3-82d3-4615-a418-e9000949275e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.851454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a55d3-82d3-4615-a418-e9000949275e None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.851808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 220/459] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:16.861272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7eb7-287f-49f6-a632-65f78693ec1b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.861272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7eb7-287f-49f6-a632-65f78693ec1b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.861272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7eb7-287f-49f6-a632-65f78693ec1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.889211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-28ea7eb7-287f-49f6-a632-65f78693ec1b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:16.890511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-28ea7eb7-287f-49f6-a632-65f78693ec1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:16.939436 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d915bf-456e-411a-a4fb-5a8eb414af66 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:16.955587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:16.975948 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:16.976629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.976916 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.977197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.977964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:16.978272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:16.979017 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 221/460] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:16.979750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:16.980376 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9596a39-6c0b-4a92-9bc5-99fe8fa24a0a None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.980670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9596a39-6c0b-4a92-9bc5-99fe8fa24a0a None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.980948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9596a39-6c0b-4a92-9bc5-99fe8fa24a0a None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.982505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 222/461] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:16.988024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:16.988323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:16.988601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:16.989064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:16.993815 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 240/462] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:17.002187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.014346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:17.015613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:17.098991 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 223/463] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:17.099871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.150868 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c500b-ef8b-4b7d-a1f4-51b8c9d98523 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.190220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.190937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.191215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.191493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.192250 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:17.193891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:17.222291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 241/464] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:02:17 2022] POST /identity/v3/users => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:17.223491 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.238459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef3abf-70ab-4a7e-b9ff-795584c31464 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.258028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:17.305935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.306788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.307068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.307346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.330610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:17.332111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:17.356855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:17.370970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:17.372062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:17.373079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:17.373079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:17.373079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:17.373079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:17.374955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 224/465] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:17.380834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.439129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db258e90-58ca-4887-8fc2-f5dcde45820f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.484983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.485857 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.486140 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.486469 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.499633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:17.501124 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:17.517767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fb0b6-20be-48fd-93c8-8d3b7f72ff81 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-4d975af5-ca52-4f49-8420-42b48071dcf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:17.520531 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:17.521619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:17.522630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:17.522630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:17.522630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:17.522630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:17.522630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 242/466] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:17.524025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071eab-c15c-4d70-a9f4-6dbb0c63c90a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.524025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071eab-c15c-4d70-a9f4-6dbb0c63c90a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.524025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071eab-c15c-4d70-a9f4-6dbb0c63c90a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.529331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 243/467] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:17.531907 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.532426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.533452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.555094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:17.556493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:17.561469 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.563372 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.563372 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.563372 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.563659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:17.563877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:17.565300 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:17.577869 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 225/468] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:17.578805 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.611609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:17.636950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 244/469] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:17.638150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.689126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:17.731548 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b9a69-a83f-4e0a-918f-6c58ece9f440 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.759985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6ff2-d8ed-4b7e-b207-830c971d3b5a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.772810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.773612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.773900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.774183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.774931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:17.775235 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:17.776743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:17.795052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 226/470] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:17.795998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.799705 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.800586 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.801112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.801556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:17.802618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:17.804239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:17.827789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 245/471] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:02:17 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:17.829146 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:17.934833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:17.946438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:17.954550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff5abd-a503-470f-bd98-39644e8b8498 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.962795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc89b3-7ed7-4ebe-b8d5-4d8dfcfc56ac None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:17.997006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:17.998096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:17.998535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:17.999048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.000213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:18.002528 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:18.002785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:18.003518 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.003878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.004228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.016364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:18.018300 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:18.041372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 246/472] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:02:17 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:18.042286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:18.043266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:18.044669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:18.044669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:18.044669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:18.044669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:18.044669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 227/473] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:18.050150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:18.208225 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bc326-7555-4fa5-81a3-22358264d119 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:18.248010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:18.249140 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.249750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.250232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.251229 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:18.251767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:18.253522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9e54-c57c-4daa-af68-0e561de51091 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:18.269474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 247/474] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:18.277740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:18.390815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb9f06-a563-4175-8021-9157194065d6 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:18.428011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:18.428696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.428983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.429283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.430018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:18.431473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:18.439514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:18.452719 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5251-2fc0-416e-8bad-16eca32ce400 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.453740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5251-2fc0-416e-8bad-16eca32ce400 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.454193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e5251-2fc0-416e-8bad-16eca32ce400 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.455838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 248/475] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:18.462940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.463355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.463792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.464183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 228/476] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:02:18 2022] POST /identity/v3/users => generated 249 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:18.484750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:18.486298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:18.490429 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:18.575874 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 249/477] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:18.624605 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:18.744122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f32c5a-e30f-4256-bd61-08cae3811be3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:18.784389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:18.785134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.785497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.785778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.786602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:18.788025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:18.797076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.sql.core [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 15:02:18.797076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 15:02:18.797076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [parameters: {'id': 'af38b8d7dfe94da08a0cf52bbbaea17b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 15:02:18.797076 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277130) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result = self._query(query) May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application conn.query(q) May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result.read() May 19 15:02:18.814618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return ref.to_dict() May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 15:02:18.815706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application next(self.gen) May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application yield resource May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application next(self.gen) May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.session.rollback() May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.force_reraise() May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise self.value May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application session.commit() May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 15:02:18.816815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._prepare_impl() May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.session.flush() May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._flush(objects) May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application compat.raise_( May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application flush_context.execute() May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rec.execute(self) May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 15:02:18.818636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 15:02:18.819876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result = self._query(query) May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application conn.query(q) May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result.read() May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 15:02:18.825324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [parameters: {'id': 'af38b8d7dfe94da08a0cf52bbbaea17b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:18.826678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 15:02:18.827977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:18.829249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 229/478] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 15:02:18 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:18.829347 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:18.935173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbb7d7-3ce3-4e8e-8954-4a97a49715fc None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:18.970590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989a8a0-ba65-453c-85b9-f04dd96a0e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:18.971426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989a8a0-ba65-453c-85b9-f04dd96a0e3f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:18.971708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989a8a0-ba65-453c-85b9-f04dd96a0e3f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:18.971981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989a8a0-ba65-453c-85b9-f04dd96a0e3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:18.980052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:18.982903 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989a8a0-ba65-453c-85b9-f04dd96a0e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f989a8a0-ba65-453c-85b9-f04dd96a0e3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:18.991813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:18.994062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:18.996081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 250/479] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:19.001260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:19.060810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:19.123477 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546901-92df-4317-b102-8cb3be1a8553 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:19.175703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:19.176692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:19.177252 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:19.177722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:19.179170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:19.179512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:19.180928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d439-e0a5-4b5c-8053-688ade219641 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:19.190496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 230/480] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:19.943795 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb81d-c827-49de-87ad-ba769a1748bc None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:19.944377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb81d-c827-49de-87ad-ba769a1748bc None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:19.944639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb81d-c827-49de-87ad-ba769a1748bc None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:19.946422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 251/481] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:19.951796 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:19.952234 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:19.952470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:19.974248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:19.975461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:20.044784 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 231/482] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:20.080266 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64beb81d-c827-49de-87ad-ba769a1748bc None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:20.228344 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64beb81d-c827-49de-87ad-ba769a1748bc None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:20.282049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:20.282911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.283317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.283601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.298496 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:20.306338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:20.307392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:20.308340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 252/483] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:20.313980 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:20.397558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:20.458889 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591bdf5-f803-4504-b0ca-98a706e2985e None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:20.508801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:20.509789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.510248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.510698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.511622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:20.512094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:20.517759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:20.532413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 232/484] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:20.538224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:20.632013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:20.685325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470b09-7885-4906-af6f-c100684f759c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:20.732100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:20.732948 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.733257 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.733583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.755538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:20.757062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-ca71594f-2e8b-4d70-ae2a-8b5c1ca562ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:20.775358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:20.776488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:20.777527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:20.777527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:20.777527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:20.777527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:20.777527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 253/485] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:20.783599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:20.807633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b9d56-62ad-40b9-9307-10dac528ffde None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.807988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b9d56-62ad-40b9-9307-10dac528ffde None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.808197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b9d56-62ad-40b9-9307-10dac528ffde None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.809628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 254/486] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:20.819008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3be59-2073-4702-84da-0898e6e0fbab None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.819296 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3be59-2073-4702-84da-0898e6e0fbab None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.819569 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3be59-2073-4702-84da-0898e6e0fbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.838479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-c4a3be59-2073-4702-84da-0898e6e0fbab None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:20.839849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-c4a3be59-2073-4702-84da-0898e6e0fbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:20.932891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3913a-0b00-4af7-8146-6ca263330139 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:20.944191 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:20.957315 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 255/487] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:20.959110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256ae9c-79c2-45c5-aed0-75d566c62b29 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.959406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256ae9c-79c2-45c5-aed0-75d566c62b29 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.959692 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256ae9c-79c2-45c5-aed0-75d566c62b29 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.966080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 256/488] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:20.967465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82647-122c-464f-bd95-3df530494828 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.967890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82647-122c-464f-bd95-3df530494828 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.968171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82647-122c-464f-bd95-3df530494828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.988269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-21c82647-122c-464f-bd95-3df530494828 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:20.989534 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-21c82647-122c-464f-bd95-3df530494828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:20.992057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:20.992836 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:20.993117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:20.993433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:20.994231 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:20.994543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:20.995935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.008879 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 233/489] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:21.009748 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.039582 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.093259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 257/490] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:21.094195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c82647-122c-464f-bd95-3df530494828 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.131979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18886bef-b1a3-40e3-b00e-57ecfa8b4d34 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.171145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.172438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.172938 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.173456 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.185551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:21.193732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:21.194767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:21.195718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:21.198248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 234/491] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:21.199269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.211336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c82647-122c-464f-bd95-3df530494828 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.236740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.259685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.261179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.262134 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.262627 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.273741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.276188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:21.299873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:21.300976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:21.302131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:21.303416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 258/492] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:21.303764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.323123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229312ed-b8b0-4413-b204-b1b947c91c7b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.344868 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.361407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.362366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.362580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.362833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.373887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:21.381814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:21.382883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:21.383852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:21.388051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 235/493] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:21.388859 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.429267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2719bf5-c9dc-495e-9819-bf68579de892 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.467252 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.468426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.468925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.469493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.470555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:21.471099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.472839 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.482291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 259/494] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:21.483129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.484629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.497686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29707f-234e-4e71-946b-235d38b61ffd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.534926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.535608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.535882 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.536159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.537008 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:21.537353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.538801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.575888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 236/495] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:21.576706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.592530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.600827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999955a-162e-497b-a4bd-b3a15000e67d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.638475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.639161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.639448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.639751 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.640542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:21.640853 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.642675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561186c3-4f5d-4ddd-936e-34d2d7894c7a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.655280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 260/496] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:21.655280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cbbae-c24b-4549-ab19-c936791ee610 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.655280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cbbae-c24b-4549-ab19-c936791ee610 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.655280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cbbae-c24b-4549-ab19-c936791ee610 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.655280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 261/497] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:21.655280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cbbae-c24b-4549-ab19-c936791ee610 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.686815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4f35-7e61-4684-921f-1188a8f44879 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.722484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.723308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.723597 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.723876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.736382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.737885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.747104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:21.758167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:21.759199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:21.760242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:21.760242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:21.760242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:21.760242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:21.760242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 237/498] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:21.760242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:21.770843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cbbae-c24b-4549-ab19-c936791ee610 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.807421 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.808211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.808424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.808735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.835957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:21.871705 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4cb4-a6cc-4c58-aafa-d26b5c917b9b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:21.882655 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.884175 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.906356 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:21.907161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.907440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.907711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.918913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:21.920374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-4e388615-0a4c-4e13-9bba-69f5bd340697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:21.937699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:21.938716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:21.939736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:21.939736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:21.939736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:21.939736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:21.939736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 238/499] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:21.940340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:21.940436 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:21.940686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:21.960629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:21.962098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:21.998319 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 262/500] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:21.999260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.017507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.040094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 239/501] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:22.040959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.071327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.121979 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41908bee-ad58-4d07-8882-b13fb66b5100 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.146284 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b086-af61-42ea-a5d0-0c1fa7043c35 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.159453 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.160142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.160425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.160917 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.161885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:22.162284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.163952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.176934 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 263/502] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:22.177904 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.190253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.190891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.191177 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.191452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.192184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:22.192489 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.193913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.210105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 240/503] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:22.210105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.259149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.292323 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9341abc-b3f6-48d5-90bd-7049f27bba2f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.309025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.321190 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620a8cd-a921-4ae4-a49b-46f1e864145e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.328056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.328871 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.329154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.329529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.340159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.341727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.354761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.355558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.355843 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.356117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:22.358686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:22.359843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:22.361425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:22.361425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:22.361425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:22.361425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:22.361425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 264/504] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:22.361425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.368138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:22.375984 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:22.377097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 241/505] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:22.383343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.460371 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338aead-917e-4511-bf25-8c4e74bd1159 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.485972 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08a618-d1f6-4bb5-ab37-d84f03f5a140 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.487735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.493935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.494736 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.495048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.495327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.502903 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.504368 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-81682dc0-e758-4734-8f1d-d3ade560df67 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:22.518680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:22.519697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:22.520820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 265/506] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:22.522110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486fc9-4888-428b-9d08-f1ee1f7bb2d2 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.522110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486fc9-4888-428b-9d08-f1ee1f7bb2d2 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.522110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486fc9-4888-428b-9d08-f1ee1f7bb2d2 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.525349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 266/507] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:22.525492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.525635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486fc9-4888-428b-9d08-f1ee1f7bb2d2 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.525732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.525732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.525732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.533970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.535408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:22.551424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:22.552687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 242/508] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:22.553931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.556214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.639410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486fc9-4888-428b-9d08-f1ee1f7bb2d2 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.658602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395910d-ea39-490b-807e-abd684c64088 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.673527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.674197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.674477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.674757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.675493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:22.675799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.677198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.689092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 267/509] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:22.689919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.692753 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.693568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.693953 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.694355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.695117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:22.695428 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.696818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e8d3b-3098-440f-addd-7aa18ef4a1e6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.705464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 243/510] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:22.706665 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.706957 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.707237 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.723817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:22.725003 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:22.743149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.777113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:22.789707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6a9e0-d655-4b66-9884-b4a923206b7b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.794756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 244/511] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:22.795599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.824880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.825604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.825880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.826146 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.826988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:22.827331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.828820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.865203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 268/512] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:22.866410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.903254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729fd05-7bfa-444f-9e9e-a83276ce904b None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.937786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:22.938454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:22.938736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:22.938995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:22.939840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:22.940149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:22.941610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:22.958647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 245/513] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:22.959432 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:22.970530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f366a7-359f-46af-a6bd-78514a5fa90d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:22.973394 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.004792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.005628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.005905 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.006178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.012786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.014290 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:23.029097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:23.030400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:23.031450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 269/514] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:23.032113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.040262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.062836 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb183bc-b65d-4a4c-b09a-835a5a9ebeda None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.096712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.097542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.097823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.098097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.108198 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.109820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:23.127834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:23.128877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:23.129918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:23.129918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:23.129918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.129918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:23.129918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 246/515] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:23.129918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.130898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89352c-6794-4ca5-aefd-e5ad4059f2d9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.166791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.167529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.167814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.168091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.178771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.180410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.190809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:23.197646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:23.198732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:23.199755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:23.199755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:23.199755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.199755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:23.199755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 270/516] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:23.199755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.232735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f748a-b790-4bc7-a878-027e5e0d12e0 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.257340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.271708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.272395 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.272675 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.272956 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.302830 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1210f1-7411-479b-89c3-540f4ceaf895 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.337033 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.337721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.338015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.338295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.345821 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.347300 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.406191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.407586 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.407726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.445916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 247/517] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:23.446688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.467797 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.511760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 271/518] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:23.512592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.550968 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cd7b1-a940-4c5b-9069-62ff35f6baa5 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.584902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.585604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.585885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.586163 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.586949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:23.587259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.588712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.606296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 248/519] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:23.607094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.609445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cb7ca-2722-42a4-aa9c-2737f06bdcae None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.633340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.643871 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.644544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.644828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.645111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.645867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:23.646177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.647579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.659102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 272/520] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:23.659977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.678512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.713261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc848-e19b-4879-935e-d61960dea65f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.746369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.748234 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.748528 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.748808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.749666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:23.749986 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.751368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.758487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61b0a3-c1de-4da5-8b49-5243447d61d2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.762660 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 249/521] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:23.763439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.792993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.793833 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.794116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.794392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.804806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.806289 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:23.823343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:23.824501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:23.825581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:23.825581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:23.825581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.825581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:23.825581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 273/522] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:23.825894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.853644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.869457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d65bd-0eeb-474b-ab40-52cc54cb192c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.896847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:23.903495 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.904699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.904986 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.905299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.915856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.917362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.927586 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4876d-5d7b-4731-9f49-47d3479e359f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:23.934416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:23.935567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:23.936728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:23.936728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:23.936728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:23.936728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:23.936728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 250/523] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:23.936728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:23.969683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:23.970253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:23.970537 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:23.970817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:23.971671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=b764ac4c637749019a830d8cf3a22e09) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:23.971992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:23.973521 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:23.979652 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 274/524] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/role_assignments?user.id=b764ac4c637749019a830d8cf3a22e09&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:23.980445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.059918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274362f5-7b8b-4d99-aa12-63b6c36c422a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.099216 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.100234 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.100521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.100800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.104839 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.111008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cb49a-8cee-458c-814e-abccb4ad4a56 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.111608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.112948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:24.129041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:24.135433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 251/525] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:24.136582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.158909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.159592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.159879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.160158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.160887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.162423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.169750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.186326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 275/526] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:02:23 2022] POST /identity/v3/users => generated 247 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:24.187379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.259222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761992-86f7-4fd4-8e1d-f280818373dd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.306743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.307430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.307720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.308003 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.308767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:24.309138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.310691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.325149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 252/527] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:24.326191 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.349591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f2b5-8235-4c29-85d3-397fe90acbfc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.396145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.419316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.420731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.420731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.420731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.422865 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:24.422865 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.423635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.438159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 276/528] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:24.439130 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.454584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e602-73f3-4cac-b4f5-e21d5fc5c840 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.501272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.501272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.501272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.501272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.501740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:24.501740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.501740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.506941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.526964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 253/529] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:24.527746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.567465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7f82f-225d-4363-a9fe-b22f842b5c08 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.602277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.603093 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.603375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.603651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.610741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.612228 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:24.626336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:24.627350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 277/530] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:24.628755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.648044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaf83a-95b7-4f75-8739-a1c6348d8838 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.665338 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.689310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.690177 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.690485 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.690832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.698858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.700355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:24.716526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:24.718848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 254/531] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:24.721089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.735527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf1abf-11cb-40da-b20f-b60aaf2df47c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.746465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.775043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.775711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.776305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.776305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.837736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db891-c6a5-4e0d-be35-e3b9554d4e0c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:24.849123 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.850835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.877061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:24.878326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:24.878915 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:24.879387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:24.880469 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:24.881061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:24.882739 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:24.901624 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 255/532] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:24.902857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.940802 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:24.959366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 278/533] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:24.959366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:24.967838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:25.022758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce370-f612-4170-a4dc-024acb87f24e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.057427 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492293e3-d5cd-4815-9130-6c03ba5739c3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.061681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.062588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.063064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.063564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.064555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:25.065057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.066838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:25.080780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 256/534] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:25.082103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:25.093604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.094207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.094500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.094774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.095616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=d76042c4941040f1835e50410d18e760) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:25.095931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.097431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:25.109270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 279/535] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:24 2022] GET /identity/v3/role_assignments?user.id=d76042c4941040f1835e50410d18e760&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:25.109270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:25.189862 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:25.217430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07921de7-9bd4-4d1b-9695-d256348d08c3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.251635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.252393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.252683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.252964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.253856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=222980e0898e4d429af49ed80c7b4b60) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:25.254174 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.254367 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:25.255668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:25.264453 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 280/536] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:25 2022] GET /identity/v3/role_assignments?user.id=222980e0898e4d429af49ed80c7b4b60&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:25.271535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7103c31-dedc-411b-864c-b5ffc53579f3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.338611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.339465 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.339746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.340028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.357710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.359254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:25.388777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:25.390015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:25.391041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:25.391041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:25.391041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:25.391041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:25.391041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 257/537] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:25.397128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:25.537941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995439f-c6c0-4a7f-ba16-453fe783cbc9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.574170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.575191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.575678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.576137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.577056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:25.577679 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.579269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:25.586962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:25.596855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 281/538] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:25.597944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:25.748332 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225be3-4d63-4c60-a0b1-b8449499383c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.796445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.797312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.797657 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.797943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.810892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.812541 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:25.829728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:25.830773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:25.831789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 258/539] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:25.836844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:25.948507 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8e75-a73e-4c22-9248-73979995c247 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:25.983442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:25.984127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:25.989650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:25.989943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:25.990789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:25.991111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:25.992669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16dd17-e23a-479b-ac3c-00d93a0d6e54 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:26.014951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 282/540] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:26.021042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:26.030421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:26.138706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12053-7072-47e3-8703-a7ebaf38c52a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:26.175613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:26.176273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.176556 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.176840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.177727 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=abe824c999d34757b0c84e96bfa3d1a6) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:26.178052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:26.179518 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f349ea-b59d-4c0b-9d5e-436ace2b1875 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:26.185685 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 259/541] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:26 2022] GET /identity/v3/role_assignments?user.id=abe824c999d34757b0c84e96bfa3d1a6&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:26.316096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5e4de-8564-478d-9561-c60efbcbaa35 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.316702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5e4de-8564-478d-9561-c60efbcbaa35 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.316702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5e4de-8564-478d-9561-c60efbcbaa35 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.318528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 283/542] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:26.324633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.325343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.325343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.350379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:26.351639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:26.447571 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:26.454550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 260/543] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:26.485594 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5e4de-8564-478d-9561-c60efbcbaa35 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:26.575271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:26.635079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5e4de-8564-478d-9561-c60efbcbaa35 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:26.689977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59f239-4a62-4d7d-9038-e0fd2b79bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:26.690816 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59f239-4a62-4d7d-9038-e0fd2b79bec0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.691101 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59f239-4a62-4d7d-9038-e0fd2b79bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.691380 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59f239-4a62-4d7d-9038-e0fd2b79bec0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.705277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59f239-4a62-4d7d-9038-e0fd2b79bec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-da59f239-4a62-4d7d-9038-e0fd2b79bec0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:26.712993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:26.718561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:26.719591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:26.722890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 284/544] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:26.728902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:26.863384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998605-e1d0-4e13-9881-bda5de8c812f None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:26.907022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9742578-b7e5-4d8d-9f0d-f1931fe41ffc None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.907684 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9742578-b7e5-4d8d-9f0d-f1931fe41ffc None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.908181 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9742578-b7e5-4d8d-9f0d-f1931fe41ffc None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.910198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 285/545] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:26.921543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:26.922156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.922273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:26.922509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.922788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.923223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:26.923558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:26.923871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:26.924144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:26.925277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:26.940037 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 261/546] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:26.942724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:26.944179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:26.945912 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.052843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 286/547] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:27.055842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:27.088935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.114015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf66443-347f-4e31-9254-7f66cda9059a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.167979 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:27.178566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:27.179381 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:27.179662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:27.179937 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:27.197044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:27.198549 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:27.230652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:27.231694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:27.232704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:27.232704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:27.232704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:27.232704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:27.232704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 262/548] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:27.237816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.274201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc45d2-2413-4076-9cf1-e6a159455643 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.337473 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:27.338299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:27.338581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:27.338856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:27.361494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:27.372831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:27.374027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:27.374987 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:27.381269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 287/549] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:27.389136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.423778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6e177-5ac0-4b54-9728-b25ac9f493d7 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.445393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:27.486874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:27.488356 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:27.488838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:27.489664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:27.491180 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:27.491966 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:27.494731 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:27.523117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 263/550] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:27.528761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.573989 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38985ffe-e222-4627-a9ee-005187d479b9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.597632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:27.633825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:27.634438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:27.641465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:27.642154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:27.642967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:27.643392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:27.644914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:27.660850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 288/551] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:27.668223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.680205 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e4d9-6793-43e3-a0fa-edc843bca770 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.715724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:27.716527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:27.716811 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:27.717094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] PATH_INFO: `/v3/users/nova` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:27.724949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:27.726466 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:27.741649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:27.742756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:27.743835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 264/552] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:27.754537 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:27.820009 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:27.857842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c37a5-083f-4f28-bb63-d3b44b6aaaa9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.918403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:27.919234 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:27.919513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:27.919790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:27.937150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:27.940163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:27.953816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebf719-f6b5-4b59-90c4-40303814cbfd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:27.972550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:27.973610 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:27.974699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:27.974699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:27.974699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:27.974699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:27.977274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 289/553] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:27.980739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.024300 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.025151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.025644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.025954 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.027593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:28.027593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.028839 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf8bc6-9e85-4462-b391-63e03287ebec None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.035781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.060773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 265/554] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:28.062872 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9296-6ddf-405b-9775-165ec14f651c None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.063359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9296-6ddf-405b-9775-165ec14f651c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.063830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f9296-6ddf-405b-9775-165ec14f651c None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.069578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 266/555] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:28.070739 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9296-6ddf-405b-9775-165ec14f651c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.134925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99c2e1-c0fd-4780-ab35-40cd42495756 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.173362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09625274-0235-4401-95c0-534749e8e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.173833 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09625274-0235-4401-95c0-534749e8e299 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.174104 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09625274-0235-4401-95c0-534749e8e299 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.174381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09625274-0235-4401-95c0-534749e8e299 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.175174 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09625274-0235-4401-95c0-534749e8e299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:28.175604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09625274-0235-4401-95c0-534749e8e299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.177188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09625274-0235-4401-95c0-534749e8e299 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.190708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 290/556] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:28.192240 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec1d4-dcd7-4821-9d6a-d8aca53c3c58 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.192465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec1d4-dcd7-4821-9d6a-d8aca53c3c58 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.192745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec1d4-dcd7-4821-9d6a-d8aca53c3c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.193809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.206510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f9296-6ddf-405b-9775-165ec14f651c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.213120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-f01ec1d4-dcd7-4821-9d6a-d8aca53c3c58 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:28.214438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-f01ec1d4-dcd7-4821-9d6a-d8aca53c3c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:28.249228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.250161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.250447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.250722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.262124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.263577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:28.285278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:28.286578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:28.287940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:28.287940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:28.287940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:28.287940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:28.287940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 267/557] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:28.287940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.294546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 291/558] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:28.296402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5b64-b434-484b-b249-02f054e1bfd3 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.296402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5b64-b434-484b-b249-02f054e1bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.296628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea5b64-b434-484b-b249-02f054e1bfd3 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.302012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 292/559] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:28.302794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea5b64-b434-484b-b249-02f054e1bfd3 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.318713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.403539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ba14-0e17-4b35-a7e9-0243d1a0d387 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.410642 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea5b64-b434-484b-b249-02f054e1bfd3 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.446601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.447328 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.447615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.447894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.449081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.452836 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.455138 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.455671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.458173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:28.458523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.460454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989fe77-82ce-4e51-968a-2b161c664df3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.462405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.464083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.472880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 293/560] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:28.474294 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.474589 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.474870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:28.481399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:28.482664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:28.483905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:28.490790 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 268/561] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:28.490790 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.494107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:28.494558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.495161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:28.570121 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 294/562] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:28.570870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.573330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.616953 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb73e3-1616-4b15-ba0d-8c35bb26ca44 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.653176 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c579f-d136-4fda-b392-16990360638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.654024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c579f-d136-4fda-b392-16990360638f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.654320 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c579f-d136-4fda-b392-16990360638f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.654595 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c579f-d136-4fda-b392-16990360638f None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.666930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c579f-d136-4fda-b392-16990360638f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-382c579f-d136-4fda-b392-16990360638f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:28.675078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:28.676369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:28.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 269/563] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:28.678251 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c579f-d136-4fda-b392-16990360638f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.694411 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c1204-00f0-4965-a764-2cde89b7be6d None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.732686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.733511 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.733796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.734070 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.741745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.743332 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.744035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:28.759254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:28.760355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 295/564] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:28.761821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.790293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c579f-d136-4fda-b392-16990360638f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.826880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.827173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.827619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.827917 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.828202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.829053 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:28.829431 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.830869 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.850013 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 270/565] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:28.850987 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.875806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8910583-efbf-44e9-93f3-1bfaff903267 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.910593 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.911382 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.911665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.911936 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.922092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:28.929746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:28.930929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 296/566] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:28.932365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:28.949589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2728195-8109-496b-a6f8-4a5d28747554 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:28.973490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:28.985605 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:28.986199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.986484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.986755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:28.987516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:28.987821 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:28.989196 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad21b9-8234-4ecd-9bf1-7135fd74757e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:28.997660 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 271/567] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:28.998709 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363331c-75cd-45c8-a054-b27faef6fcf2 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:28.999062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363331c-75cd-45c8-a054-b27faef6fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:28.999350 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363331c-75cd-45c8-a054-b27faef6fcf2 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.000587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 272/568] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:29.001431 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363331c-75cd-45c8-a054-b27faef6fcf2 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.041353 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a6cda-5819-4d06-b840-7347f8452873 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.064971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.076858 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.077570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.077855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.078132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.078925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:29.079234 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.080693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.092672 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 297/569] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:29.098132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.106818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363331c-75cd-45c8-a054-b27faef6fcf2 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.141061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d95e-73c3-4a15-8652-572c463edf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.141745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d95e-73c3-4a15-8652-572c463edf6a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.142381 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d95e-73c3-4a15-8652-572c463edf6a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.142658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d95e-73c3-4a15-8652-572c463edf6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.194608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.199631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cee41-ab87-4f6e-bddf-e1bde46ec7c4 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.211681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37d95e-73c3-4a15-8652-572c463edf6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.213148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37d95e-73c3-4a15-8652-572c463edf6a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.234498 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.235167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.235452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.235783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.236487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:29.236797 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.238209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.246240 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 298/570] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:29.247041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.297200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.318618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 273/571] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:29.319935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.320253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.320512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.337553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:29.338803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:29.354793 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfdc45-8680-41e5-beb5-f41f285df48c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.388220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.389021 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.389322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.389666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.400671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.402168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.403921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 274/572] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:29.404726 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.419458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:29.428019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:29.429822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:29.431313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:29.431313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:29.431313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:29.431313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:29.431313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 299/573] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:29.431313 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.510044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1323c0-6810-4763-b1e9-9b50c7430812 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.528126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.535403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0fce2-db42-4808-963b-fb08dfda3c4e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.551725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.552630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.552912 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.553185 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/b764ac4c637749019a830d8cf3a22e09/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.568342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=b764ac4c637749019a830d8cf3a22e09, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.569834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.572760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.573684 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.573896 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.574170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.583409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 275/574] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:29 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/b764ac4c637749019a830d8cf3a22e09/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:29.584214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.585258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.586829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:29.607960 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:29.609326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:29.610550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:29.610550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:29.610550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:29.610550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:29.612301 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 300/575] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:29.613398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.667650 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.714260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cda646-00fa-4d21-bc53-9c1fee57a10c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.727487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ec19-356d-40a2-ae59-b26303d9fd21 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.757250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.758105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.758393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.758681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.766102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.767199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.767483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.767758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.772935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.773862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.783422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:29.792435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:29.793986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:29.795682 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 301/576] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:29.796637 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:29.797159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:29.798902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:29.802304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:29.802304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:29.802304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:29.802304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:29.803713 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 276/577] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:29.805273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.811074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.908349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83e02e-88e1-49f3-8654-255b86eb26e1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.921327 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3fafb-7fb0-48bf-8383-0d8441ff6d0b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:29.938850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:29.944475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.945199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.945497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.945748 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.946486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:29.946795 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.948214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.960082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:29.965374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 302/578] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:29.965539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:29.965539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:29.965539 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:29.965817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:29.965916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:29.965916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:29.965916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:29.980745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 277/579] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:29.980745 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.043224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.078165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef21d3-974d-4216-8f3f-49e861e51a12 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.092693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdef18-ae92-47e0-9931-a877ef1100e3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.116325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.117175 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.117488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.117761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.119145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:30.119717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.121602 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.130781 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.131453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.131740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.132017 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.132752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:30.133062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.134476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.137195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 303/580] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:30.137926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.152608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 278/581] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:30.153617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.191909 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.251939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af37f43-2452-489e-b3a0-50109f5ad762 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.261983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cc1ba-e7f4-4786-8e25-1e25b9c01c73 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.291693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.299251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.300014 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.300298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.300576 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.302324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.303531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.303820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.304094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.308519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.310035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.312587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.314049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:30.325882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:30.327154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 15:02:30.328356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:30.329047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:30.330149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 279/582] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:30.331670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.333770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 304/583] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:30.335041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.441166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.443852 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95497413-da13-4095-b0e6-af99ca957579 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.449184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ffaab-fdc9-43e8-b2e7-8ed61bbdd868 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.480593 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.481447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.481733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.482069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.484963 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.485787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.486072 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.486347 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.488890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.490395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.497636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.499141 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:30.504463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:30.505506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 305/584] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:30.511279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:30.525952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:30.527412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:30.528417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:30.528417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:30.528417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:30.528417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:30.528417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 280/585] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:30.528417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.565899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.619052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07add59-d4cf-416d-b0ea-d18502d74b45 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.635564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d923f-1eb4-47aa-a67c-c25c24b3b75a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.654169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.654880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.655150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.655456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.656294 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:30.656573 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.658082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.673544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.674498 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.674982 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.675447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.676361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 306/586] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:30.676710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:30.677180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.677404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.679057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.703117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 281/587] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:30.704004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.713664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.785691 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ccf18-0bf5-417d-9032-64a1acee010d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.796122 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.822382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7eb50-ec27-46b8-9af0-10af87885029 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.822585 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.823063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.823350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.823621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.824430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:30.825480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.826275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.840343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 307/588] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:30.842121 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.859210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.859882 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.860167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.860471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.861203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:30.861558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:30.862974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:30.875625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 282/589] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:30.876922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:30.948009 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:30.951787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6369-97af-41b0-92d6-f46473002f8b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:30.988494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:30.989339 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:30.989626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:30.989901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:30.990441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7f1f2-c066-419a-87dc-008af37e33d7 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.000776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.002294 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:31.019935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:31.021011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:31.022063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:31.022063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:31.022063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:31.022063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:31.022063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 308/590] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:31.022063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.032434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.033265 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.033544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.033817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.044741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.046278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.047604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:31.064524 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:31.065604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:31.066598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:31.066598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:31.066598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:31.066598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:31.069373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 283/591] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:31.071009 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.130472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e037708-0adf-485b-9fc4-f93f39bc121a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.164873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.165571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.165851 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.166125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.166965 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=6f8c56047dbd40928ac7cb29bbc767ba) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:31.167268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.168716 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.174639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 309/592] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/role_assignments?user.id=6f8c56047dbd40928ac7cb29bbc767ba&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:31.175720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0b689-8c29-4c01-a3af-c47edf601d4a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.178121 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.199370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.227698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.229167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.229724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.230238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.240119 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.242221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:31.259282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:31.260417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 15:02:31.261509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:31.264189 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 284/593] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:31.265384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.300390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.318688 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376c58e-b2f1-4df2-9f28-34ddd0a927b4 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.358235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.359084 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.359525 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.359869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.360711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:31.361082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.362634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.376508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 310/594] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:31.377839 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.396308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c868b58-cabd-4ee5-bb22-c11d7fa84fa6 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.435268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.435997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.436283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.436735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.437512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:31.437825 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.439222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.456799 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 285/595] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:31.457656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.499855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.505530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d1258-1903-4eed-9b8a-4cef9a09f467 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.549281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.549281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.549281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.549281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.549770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:31.549770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.549911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.572469 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69154c-feb4-438c-8e30-ee74befdf231 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.573040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 311/596] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:31.574104 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.604115 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.614464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.615279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.615568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.615853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.623145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.624666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:31.642317 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:31.643415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:31.644433 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:31.648698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 286/597] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:31.649709 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.696325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a198-3f14-4afe-bb71-8227ca018e67 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.740340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.741170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.741493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.741765 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.751224 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.753443 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:31.770842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:31.774953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 312/598] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:31.776599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.778564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.784201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0291-46e5-4fd8-9fa4-f31096a70422 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.824907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.825816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.826098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.826372 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.838741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.840207 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:31.858670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:31.859763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:31.861499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:31.861499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:31.861499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:31.861499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:31.861499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 287/599] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:31.861499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.866586 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:31.908890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa857e7-d965-49e5-8d2d-8b42be0362fc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:31.944390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:31.945076 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.945391 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.945666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.946495 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:31.946817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:31.948248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fec956-637b-4ffe-9358-ab1d5de8a920 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:31.960870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 313/600] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:31.965457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b250a8-303c-4ea5-8b66-971b0b6559b6 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:31.965457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b250a8-303c-4ea5-8b66-971b0b6559b6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:31.965457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b250a8-303c-4ea5-8b66-971b0b6559b6 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:31.965457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 314/601] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:31.965457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b250a8-303c-4ea5-8b66-971b0b6559b6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:31.975277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de276d9-c33c-44e2-b8c5-b7ad0fc5e56d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.009584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.010254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.010538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.010812 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.011735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:32.012058 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.013530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.026240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 288/602] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:32.027023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.041457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.088316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b250a8-303c-4ea5-8b66-971b0b6559b6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.116832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.131669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.132338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.132612 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.132894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.133644 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:32.134004 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.134476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e280c53-92c3-47db-bfc5-eedad2f345c0 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89d362-2e91-4efb-9c46-9536dd26262f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.153681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 315/603] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:32.153681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.153681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.153681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.168358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:32.169619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:32.176959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.177880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.178165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.178453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/d76042c4941040f1835e50410d18e760/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.193197 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=d76042c4941040f1835e50410d18e760, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.194716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.208590 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 289/604] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:32 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/d76042c4941040f1835e50410d18e760/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:32.209390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.269269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 316/605] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:32.269269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.332152 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.399627 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3c79-5df3-410a-81a4-b96c467e1c8f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.419746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea35ad-e76c-4e69-aa15-88f7285852ed None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.435650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.448118 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.449025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.449523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.449664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/222980e0898e4d429af49ed80c7b4b60/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.467291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=222980e0898e4d429af49ed80c7b4b60, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.468754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.468893 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.469973 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.470535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.471128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.480880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.482804 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.486628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 290/606] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:32 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/222980e0898e4d429af49ed80c7b4b60/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:32.487479 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:32.499137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:32.500230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:32.501277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:32.503750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 317/607] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:32.509418 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.618493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8874-b858-41c1-a01b-56eeb7804548 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.633220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.662545 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.663976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.664253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.664529 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.677203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:32.692872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f3d63-6802-497d-8452-38551d5ea30b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:32.697873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:32.698944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:32.700294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 291/608] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:32.705304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.785029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.786149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.786638 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.787148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.788235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:32.788778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.790538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.818915 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 318/609] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:32.821437 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfabe7-412c-4cc8-993a-e9a98047b53e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:32.824329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.862831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.869532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:32.870227 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:32.870513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:32.870791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:32.871562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:32.871870 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:32.873270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:32.885018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 292/610] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:32.890344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:32.928079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:32.950827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ce8b7-5682-41f7-bec8-0391156802e3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:33.009638 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:33.010548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.010823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.011099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/abe824c999d34757b0c84e96bfa3d1a6/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.038518 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=abe824c999d34757b0c84e96bfa3d1a6, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:33.039990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:33.043811 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c314bd-d928-4c18-91eb-54ceccc55093 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:33.069617 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 319/611] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:32 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/abe824c999d34757b0c84e96bfa3d1a6/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:33.092091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:33.093082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.093395 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.093715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.106138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:33.107647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:33.127377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:33.128445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:33.129503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:33.129503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:33.129503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:33.129503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:33.129503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 293/612] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:33.134483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:33.252475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:33.285590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4d21b-ca8c-4f4c-b813-46bd62a9062f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:33.337019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:33.337781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.338051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.338405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.339146 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:33.339454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:33.340920 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:33.354190 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 320/613] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:33.359397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:33.480651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:33.498352 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b971fc3-815b-415e-a893-cd5668249472 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:33.539682 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:33.540438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.540718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.541000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.551144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:33.552614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-b5bbfa51-29b5-4da7-916a-1ee4f2556a59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:33.572093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:33.573257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:33.574342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:33.577602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 294/614] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:02:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:33.583069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:33.680661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b0b66-23be-4a89-9ea5-c649a70d0752 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.681273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b0b66-23be-4a89-9ea5-c649a70d0752 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.681273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b0b66-23be-4a89-9ea5-c649a70d0752 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.682786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 295/615] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:33.689706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.689998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.690283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.722418 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:33.724137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:33.767824 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d987a8-5098-4638-a4d4-ff4f0bd6768e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:33.832269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:33.833307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:33.835497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:33.835975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:33.837561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:33.837561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:33.838497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 296/616] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:33.840220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:33.856072 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:33.861975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 321/617] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:02:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:33.870444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:33.883881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:33.978409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:34.006652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784cdf5-eb1a-4cbe-8654-7993a7147267 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.050262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.050926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.051213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.051488 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.103705 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970556ab-15a2-4bcd-a9a9-9385e4846fcd None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.126133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.127616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:34.190947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.191787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.192065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.192345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.208895 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:34.213906 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:34.229947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:34.231041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:34.232305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 322/618] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:34.238311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 297/619] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:34.239410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.244164 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.351077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8fe2d-31b4-4ba4-b23d-5341c5ebcf40 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.393752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:34.401970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.402799 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.403091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.403380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.421485 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.422975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:34.425844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4ec17-a200-4b9d-89fd-dac05aa9673f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:34.453632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:34.454702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:34.455708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:34.455708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:34.455708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:34.455708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:34.457419 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 298/620] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:34.468444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.470261 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.470947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.471246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.471516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.472272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:34.472584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.475981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:34.496341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 323/621] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:34.499773 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.524471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:34.661537 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dad9c-ea60-45a4-b3d4-237b58748b12 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.676638 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19bd68-8555-4e9f-af10-599704835219 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.713957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:34.728154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.728828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.729119 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.729422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.730173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:34.730484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.731881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:34.745544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.746367 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.746651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.746926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.758787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 299/622] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:34.764942 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.767719 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.769223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:34.810926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:34.812067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:34.813608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:34.813608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:34.813608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:34.813608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:34.813608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 324/623] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:34.819678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.858161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:34.875494 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82efceb-7d46-438f-b102-adec7566c8b2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.913970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.914720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.915007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.915284 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.922999 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.924482 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6728439a-3e77-449f-9b10-257545d0303e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:34.940391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:34.941690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:34.943133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 300/624] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:34.943699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6e28a-8232-4d7a-8586-ab7197323299 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.943699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6e28a-8232-4d7a-8586-ab7197323299 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.943849 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6e28a-8232-4d7a-8586-ab7197323299 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.945429 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 301/625] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:34.949867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc64ac-98ef-4150-90e5-3816955c7bf9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:34.950193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6e28a-8232-4d7a-8586-ab7197323299 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:34.985969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:34.987113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:34.987603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:34.988094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:34.989227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:34.989761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:34.991376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c2e88-9d98-42aa-821b-1fa41db16452 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.008541 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 325/626] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:35.010341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.010863 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.011346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.033525 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:35.034739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:35.061123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6e28a-8232-4d7a-8586-ab7197323299 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.062906 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.112266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.113552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.114036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.114480 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.121258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 326/627] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:35.121258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.121863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:35.123740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.125949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.147151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 302/628] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:35.147873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.166464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.244820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fd130-c0d5-4786-a764-eac8ec5d1b02 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.290658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.292447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.292967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.293475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.294156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43402a1-5e7d-4a8f-a929-a4b65cea2ede None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.303043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.304642 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:35.320184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:35.321330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:35.322533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 327/629] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:35.323213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.341409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.342802 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.343272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.344132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.350862 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.361229 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:35.371538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:35.372810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:35.374038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:35.376969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 303/630] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:35.378181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.450449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.453431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4661b6-4a69-4307-8958-dd782c731a24 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.489035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.489470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27942c3-bde4-4312-a947-58e6e629e2bc None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.489761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.490046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.490321 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.491185 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=b764ac4c637749019a830d8cf3a22e09) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:35.491474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.492935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.509287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 328/631] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/role_assignments?user.id=b764ac4c637749019a830d8cf3a22e09&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:35.509287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.536875 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.537682 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.538051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.538391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.539316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:35.539781 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.541462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ef490-cf01-44cb-b49a-f3548dbbfde0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.568784 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 304/632] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:35.570447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a678-5fad-4026-89bf-e68e12c6a08f None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.570844 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a678-5fad-4026-89bf-e68e12c6a08f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.571197 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a678-5fad-4026-89bf-e68e12c6a08f None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.577335 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 305/633] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:35.578415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a678-5fad-4026-89bf-e68e12c6a08f None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.646309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff05f0-f2d5-4bd2-b7e1-f092b69ac11d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.684893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.690298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.691430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.692067 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.692648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.693711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:35.694437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.695980 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72210d4b-212c-4c91-a993-a7b7129ba7be None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.699289 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a678-5fad-4026-89bf-e68e12c6a08f None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.706437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 329/634] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:35.708031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.708431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.708821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.726525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.729211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:35.730830 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:35.737944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.738755 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.739036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.739311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.750978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.752438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:35.773373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:35.774459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:35.775483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:35.775483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:35.775483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:35.775483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:35.775483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 306/635] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:35.775483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.818167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 330/636] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:35.819097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.895787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217fe69-0048-4011-8175-41cb4d50e2bb None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.912868 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:35.934259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea743bea-a04f-4a13-94f6-8d0382dd0498 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:35.937078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.937922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.938210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.938492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.950489 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.952417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.973054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5dda8262-651f-44a4-9d1a-e4b1ad7f1e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:35.973294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:35.974268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:35.975900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:35.975900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:35.975900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:35.975900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:35.976162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.976162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.976162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.976371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 307/637] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:35.976468 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:35.976468 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:35.976612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982fe6-47aa-4083-bd71-e452703620d9 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:35.976612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982fe6-47aa-4083-bd71-e452703620d9 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:35.976612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982fe6-47aa-4083-bd71-e452703620d9 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:35.976826 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:35.981364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 308/638] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:35.981364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982fe6-47aa-4083-bd71-e452703620d9 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:35.994217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 331/639] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:35.999403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.014740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.097118 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982fe6-47aa-4083-bd71-e452703620d9 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.105054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a055dc9-2161-4d4e-862a-f6db9809de0a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.134081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.134890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.135217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.135449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.138791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.139651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.140026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.140398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.141318 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:36.141650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.143482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ab96-dc69-40e4-b27f-895822662dc4 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.146949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:36.152706 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.156580 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 332/640] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:36.157967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.158501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.159071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:36.160883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:36.162133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 309/641] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:36.168105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.186054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:36.187308 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:36.262654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.264352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 333/642] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:36.264643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.303333 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809a19-f4d7-41b7-8fa8-0fe71767a396 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.340618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.341902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.342219 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.342600 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.350118 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.351689 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:36.367598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:36.368629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:36.369686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 310/643] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:36.370916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.391133 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08482b04-d40a-47d4-9fa3-829d49da6a35 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.422213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.429520 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.430514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.430898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.431197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.439720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.441323 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:36.455891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:36.457127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 334/644] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:36.458921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.485787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ea05f-8543-4670-ab5c-9503404a9d9b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.521815 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.522414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.522702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.522995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.523757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:36.524071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.525494 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.533786 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 311/645] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:36.534565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.548218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.573854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16336-9bd8-48f1-8644-215ae4e36e7e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.608817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.609660 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.609947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.610236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.621078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:36.628877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:36.630058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:36.631178 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:36.634872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 335/646] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:36.635710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.647712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b81aa-2e42-43b8-af91-9a44ad1589a1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.685498 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.689471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.689961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.690281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.690522 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.691422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:36.691625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.693063 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.711000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 312/647] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:36.711000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.746920 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ff9a1-e15f-4cb1-a95e-01b104f42658 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.778856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.793971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.794665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.794962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.795233 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.796156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:36.796503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.797980 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.822135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 336/648] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:36.822341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dac1c-2f1f-4048-a173-5c010d5d1b21 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.822482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.862671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.863471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.863758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.864033 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.876246 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.877769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:36.896892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:36.897978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:36.899014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:36.899014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:36.899014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:36.899014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:36.899014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 313/649] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:36.899014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:36.933994 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8cb04-98dd-421f-99ff-028385d81743 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:36.944450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:36.971394 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:36.972167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:36.972525 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:36.972875 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:36.973764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:36.974145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:36.975629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:36.983857 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 337/650] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:36.984729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.015080 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f01e7b-4c90-4202-b7ed-cc92a5b93954 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.039646 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.053003 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.054028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.054314 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.054595 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/6f8c56047dbd40928ac7cb29bbc767ba/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.069341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=6f8c56047dbd40928ac7cb29bbc767ba, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.070813 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.085728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 314/651] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:36 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/6f8c56047dbd40928ac7cb29bbc767ba/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:37.086778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.109868 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc063c-d090-4b10-a4c7-204863875486 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.156831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.158357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.158744 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.159146 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.215922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d561-8de7-4710-9d3a-55e6276cc31f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.224796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.263656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.265016 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.265807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.266126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.266255 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.266526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.267707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:37.268087 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.269954 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.289410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 315/652] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:37.290711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.339540 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.422140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf4f34-0a87-42e6-9e2a-9c9a438287e5 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.427538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 338/653] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:37.427538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.469062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.470733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.471244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.472074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.486373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.488712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.500497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:37.508593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:37.509662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:37.510667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:37.510667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:37.510667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:37.510667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:37.513048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 316/654] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:37.514069 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.544216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba67d-2471-431e-a3bc-dfe185fc51ab None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.586438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.587598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.588503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.588966 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.598864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.601169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:37.619126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:37.620249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 15:02:37.621372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:37.625325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 339/655] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:37.626372 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.636412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.661667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e9bff-2826-4254-ac86-57e563b61377 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.712234 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.713102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.713415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.713696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.732978 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.735063 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.757256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2165d-aef8-4ea4-aff2-78d420d4da83 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:37.758800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:37.759757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:37.762425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:37.762425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:37.762425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:37.762425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:37.762425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 317/656] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:37.762425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.796058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.802223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.803079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.803312 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.803680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.804455 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:37.804835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.806457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.823195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 340/657] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:37.824236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.893898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe078c2-b9a8-448f-aa82-bae9174f158f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.929614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.930289 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.930573 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.930846 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.931118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f0a85-c690-4c5e-b6af-edcbdd501ced None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:37.931688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:37.931997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.933930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.937449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:37.954599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 318/658] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:37.955440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:37.970470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:37.971158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.971438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.971711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.972434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:37.972740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:37.974153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b9a4-1c74-4fe9-92d6-cfd161ede143 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:37.990480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 341/659] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:37.990480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6851ef-44c4-44b5-87a8-bb0dd0527751 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:37.990480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6851ef-44c4-44b5-87a8-bb0dd0527751 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:37.990480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6851ef-44c4-44b5-87a8-bb0dd0527751 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:37.990480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 342/660] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:37.990480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6851ef-44c4-44b5-87a8-bb0dd0527751 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.048275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.064383 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f91c07-708a-4724-bb63-4b6fdaf86a27 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.102467 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6851ef-44c4-44b5-87a8-bb0dd0527751 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.107721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.108558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.108840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.109112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.116824 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.118312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:38.134417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:38.135455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 319/661] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:38.136530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.138870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.139667 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.140259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.140382 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.199436 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.211449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.212930 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.248436 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bd924-a5a1-45bc-8584-aee4f9e025b9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.264422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.293580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.294382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.294660 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.294930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.302068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.303593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-8f120a00-c38d-4d9c-8efd-e6d5af807522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:38.317493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:38.318622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:38.319634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 320/662] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:38.320964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.320964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.320964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.335428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 343/663] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:38.335428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.337761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:38.338939 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:38.414340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 321/664] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:38.415211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.436950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36652681-441e-41a2-a303-7b968a386e13 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.473143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.473864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.474098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.474212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.474456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.475365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:38.475896 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.477663 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.495908 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 344/665] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:38.496527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.507215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.537166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1ef20-593a-4e39-96c5-43791bbb79e5 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.573195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.573948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.574240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.574509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.575343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:38.575734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.577391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.589484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 322/666] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:38.590400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.604485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac06fb-9cc4-4e09-8bf7-4425f5be13ec None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.637295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.638093 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.638373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.638643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.650073 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.651539 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:38.670621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:38.671681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:38.672832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:38.672832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:38.672832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:38.672832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:38.674445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 345/667] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:38.675504 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.703707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb505cd-1b97-4aeb-ad72-85ee782548af None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.706280 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.738774 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.739661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.740028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.740350 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.743624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.751561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:38.759864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:38.761126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:38.762299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:38.765915 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 323/668] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:38.765915 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.791937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2148-cb3f-4020-8c45-5cb975affa41 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.826563 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.827140 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.827424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.827702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.875708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0e42a-82d9-47e6-8fbe-7f1fac17ee85 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:38.899892 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.901445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.910769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:38.911651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:38.912036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:38.912420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:38.913632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=d76042c4941040f1835e50410d18e760) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:38.914059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:38.915912 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:38.929281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 324/669] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/role_assignments?user.id=d76042c4941040f1835e50410d18e760&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:38.929281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:38.965576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:38.989395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.031543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 346/670] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:39.032681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.065376 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c939192-1e37-45ba-900f-c9b140a0b6ee None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.104603 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.105459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.105805 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.106143 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.106979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:39.107343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.109277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.128132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 325/671] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:39.129431 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.146898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9df97-b568-457c-b8c1-f8c63462b1f0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.190194 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.191209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.191668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.191952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.192754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:39.193777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.194548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.204434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 347/672] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:39.206446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.235637 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.248665 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.263700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517fe23-f033-49e0-ae7c-7adf7ea2973d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.305414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.306399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.306547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.306796 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.317827 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.319273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.328671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8a73b-6a9d-426f-bde9-b13fd95a8d98 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:39.337564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:39.338773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:39.340121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:39.340121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:39.340121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:39.340121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:39.343690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 326/673] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:39.344593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.367259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.368406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.368852 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.369312 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.376572 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.378492 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:39.394458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:39.395619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:39.396728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:39.398863 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 348/674] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:39.399845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.451157 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1829f-1a6a-4891-ae94-e6df91c6ca43 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.486544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.487325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.487453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.487617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.487881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.496704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.499457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.501376 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.510139 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07001cdc-cc21-406f-89dd-a0d3431a561e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-56cd1b89-8b1a-4613-8009-7613f1c5fe69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:39.519604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:39.520750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:39.521936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:39.521936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:39.521936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:39.521936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:39.523850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 327/675] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:39.525362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a4325-f742-40f5-abc8-bc94e9886831 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.525914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a4325-f742-40f5-abc8-bc94e9886831 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.526345 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a4325-f742-40f5-abc8-bc94e9886831 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.528141 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 328/676] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:39.528845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a4325-f742-40f5-abc8-bc94e9886831 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.558106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.558908 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.559088 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.559340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.560082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:39.560360 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.561853 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.573750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 349/677] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:39.574410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.642693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a4325-f742-40f5-abc8-bc94e9886831 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.677078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.678074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.678500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.678776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.679567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:39.679876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.681362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf27cf-a35c-4b3f-b57c-4a23eb540d9d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.683271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f22a35-2739-4a9c-a302-090d827877ec None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.697516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 329/678] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:39.697516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f544f5-2722-424a-856a-11f642058b32 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.697516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f544f5-2722-424a-856a-11f642058b32 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.697516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f544f5-2722-424a-856a-11f642058b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.713646 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-45f544f5-2722-424a-856a-11f642058b32 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:39.714790 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-45f544f5-2722-424a-856a-11f642058b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:39.729071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.729787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.730066 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.730336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.731064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:39.731369 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.732984 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.747945 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 350/679] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:39.748816 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.751504 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.774605 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:39.805264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 330/680] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:39.805264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f544f5-2722-424a-856a-11f642058b32 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.888492 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130fdff-be78-4b42-a791-af6542c430f5 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.906263 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f544f5-2722-424a-856a-11f642058b32 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:39.929151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.930003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.930293 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.930578 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.937580 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.939295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.944969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece6650-0144-4199-bd44-0930067823db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:39.945670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece6650-0144-4199-bd44-0930067823db None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:39.945960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece6650-0144-4199-bd44-0930067823db None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:39.946238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece6650-0144-4199-bd44-0930067823db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:39.947083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece6650-0144-4199-bd44-0930067823db None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=222980e0898e4d429af49ed80c7b4b60) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:39.947401 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece6650-0144-4199-bd44-0930067823db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:39.948861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece6650-0144-4199-bd44-0930067823db None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:39.955596 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 331/681] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/role_assignments?user.id=222980e0898e4d429af49ed80c7b4b60&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:39.956180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:39.957177 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:39.958237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:39.958744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece6650-0144-4199-bd44-0930067823db None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:39.960629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 351/682] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:39.961762 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.024476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.035649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.082869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404a6fc-8b05-4f52-aebd-26b19e875949 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.102543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece6650-0144-4199-bd44-0930067823db None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.142396 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.143863 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.144812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.145628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.153670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.154614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.154971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.155238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.163094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:40.165516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.167004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:40.174787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:40.175935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:40.176929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:40.178999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 352/683] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:40.179911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:40.191362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:40.192452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:40.193553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:40.195386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 332/684] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:02:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:40.196460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.305729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18255407-42fe-445d-b2c6-03bab206c5b0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.322358 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a650-808c-480d-a643-38b8c9bbd436 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.333949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.339408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.353451 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.354271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.354554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.354932 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.355737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:40.356151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.357735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.371230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 353/685] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:40.371405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.371918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.372054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.372200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.372593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.373465 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:40.373910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.376497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.387326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 333/686] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:40.389545 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.499128 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3327d-13e7-4530-9623-2ad0d222b729 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.510041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a10d3d-2f62-4768-8b0d-ae55a49f27bb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.545481 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.546030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.546279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.546533 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.546733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.546905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.547076 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.547246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.553259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=abe824c999d34757b0c84e96bfa3d1a6) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:40.553259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.553666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:40.553666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.553974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.554129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.557217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 334/687] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/role_assignments?user.id=abe824c999d34757b0c84e96bfa3d1a6&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:40.558049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.591660 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 354/688] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:40.595776 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.597476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.679523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259b45d-073f-4fee-ba07-349a7137d61f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.720144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.721343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.722499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.722795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.723068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.747615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.749532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:40.773927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:40.775071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:40.776088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:40.776088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:40.776088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:40.776088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:40.778589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 335/689] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:40.785374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:40.823535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4af5fe-8ae2-4333-959c-7a1687e6eca8 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.874783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.875958 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.876455 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.877018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.914043 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:40.933453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c41474-8a1a-434f-add0-4d92b14f7aef None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:40.963052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.965650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.976616 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:40.977812 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:40.977812 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:40.978125 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:40.978800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:40.979112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:40.980492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:40.996857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 336/690] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:41.004162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.026424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.095487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 355/691] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:41.102976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.133639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f19d6-aba6-474b-b9e6-5a8e16fad861 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.201389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.203818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.204103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.204386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.225729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.227261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:41.256430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:41.257564 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:41.258635 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:41.261430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 337/692] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:41.272315 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce687081-6419-4df4-b0e6-c88c998ea790 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.277207 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.290305 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.321535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.322434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.322718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.322997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.342123 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.344814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.346325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-a0139011-1f9c-4049-9e97-7e8f8c575c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:41.383876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:41.384968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:41.386055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:41.386055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:41.386055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:41.386055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:41.386055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 356/693] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:41.389260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3171-dfbf-4b09-bed2-d04af1a6b4f4 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.389260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3171-dfbf-4b09-bed2-d04af1a6b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.389260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3171-dfbf-4b09-bed2-d04af1a6b4f4 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.393487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 357/694] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:41.397208 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad3171-dfbf-4b09-bed2-d04af1a6b4f4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.410568 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911b291-aa6c-4ca6-aa68-7c34201e97a3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.452811 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.453649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.453940 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.454217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.455626 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:41.455951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.457741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5717-276f-497c-b7ee-868cd0599d39 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.485370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 338/695] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:41.486775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.487072 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.487356 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.509226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:41.511059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:41.522267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad3171-dfbf-4b09-bed2-d04af1a6b4f4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.561910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.562774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.563158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.563529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.564363 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:41.564818 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.566346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.579168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 358/696] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:41.580168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.586340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.604683 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 339/697] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:41.605639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.653738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.703902 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e074d9-f2c2-461d-a626-24be30422733 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.739472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.740295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.740678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.741053 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.752633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0e2a1-6c08-47b3-a237-eb36a7c528d2 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.790719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.791600 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.791995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.792366 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.799947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.801551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:41.816841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:41.818688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 340/698] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:41.820861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.826757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.828279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.880490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.902662 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:41.937832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f3226-c535-4263-a3bd-76668c41c9f2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:41.964623 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 359/699] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 15:02:41.965763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:41.975927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:41.976863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:41.977473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:41.977473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:41.978203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:41.979595 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:41.995032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:41.997005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:41.999226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:42.009292 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 341/700] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:42.009292 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.119336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33365b07-0803-44c9-adc2-aaa42d715b66 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.140678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:42.163958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.164878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.165288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.165576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.177587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.179194 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:42.198093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:42.200287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:42.202632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:42.202632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:42.202632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:42.202632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:42.202632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 342/701] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:42.202632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.210012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf8ef0-7b55-43ef-8751-4edb973018e0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.244949 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.245656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.245939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.246219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.247015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:42.247324 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.248817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.262307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 360/702] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:42.263257 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.280506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:42.326779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624f2e-ff20-46a2-badf-715859a4351e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.364447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.365124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.365442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.365724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.366453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:42.366764 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.368278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.373783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 343/703] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:42.378167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.395955 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4f703-d437-43b5-905a-255558aa6a7c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.413444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:42.441841 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.443260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.443963 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.444880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.446158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:42.446951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.449274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.464554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 361/704] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:42.466407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.491663 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837d5cc-3b05-4db0-9258-4d836df8164d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.530423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.531131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.531420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.531700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.532560 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=b764ac4c637749019a830d8cf3a22e09) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:42.532878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.534620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.541200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 344/705] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/role_assignments?user.id=b764ac4c637749019a830d8cf3a22e09&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:42.541931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.561528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:42.608796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38331b-bde7-4438-b9d9-dc4bf39b382a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.647817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.648429 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.648714 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.648994 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.649775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:42.650090 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.651629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff775-cb55-4493-85c0-be2fc634b30c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.665973 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 362/706] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:42.667644 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0460164-c0c8-48c8-ae1d-abb366aa2756 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.667948 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0460164-c0c8-48c8-ae1d-abb366aa2756 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.668232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0460164-c0c8-48c8-ae1d-abb366aa2756 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.670207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 363/707] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:42.676073 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.676421 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.676649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.681471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:42.691570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf9230-4d2d-4a06-9fa7-63b81fbae717 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.711106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:42.712373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:42.738941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:42.740212 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:42.740725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:42.741228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:42.751282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:42.753685 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:42.772385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:42.773511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:42.774532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:42.777990 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 345/708] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:42.784414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.836430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 364/709] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:42.885133 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:42.970697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c444df-8f92-464a-ba09-5cdd931e0105 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:42.977522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:43.023052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682dba6-21a7-485c-b0ea-5b3f3aa95045 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:43.030140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:43.030831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.031119 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.033270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.033270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:43.033270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:43.033995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:43.058632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 346/710] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:43.065698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:43.094505 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:43.095350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.095634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.095919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.102138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:43.122018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:43.123592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:43.156108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:43.157337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:43.158366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:43.158366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:43.158366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:43.158366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:43.158366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 365/711] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:43.164325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:43.232200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5b61e-b220-4095-b72e-1c32e769384f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:43.268592 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:43.269697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.270229 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.270734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.273854 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=6f8c56047dbd40928ac7cb29bbc767ba) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:43.274456 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:43.276254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5c6a-667d-4003-b4b6-25ea650b56dd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:43.283465 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 347/712] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:43 2022] GET /identity/v3/role_assignments?user.id=6f8c56047dbd40928ac7cb29bbc767ba&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:43.334946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c38b6f-be85-4924-a826-05f438952228 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.335644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c38b6f-be85-4924-a826-05f438952228 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.336206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c38b6f-be85-4924-a826-05f438952228 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.338259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 348/713] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:43.353192 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.353921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.354454 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.365981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b87615-039b-4ad0-a28b-83d324872b7f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:43.388078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:43.390849 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:43.396968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:43.425821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:43.426934 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.427472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.428097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.429156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:43.429724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:43.431366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:43.445804 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 366/714] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:43.452025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:43.534095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 349/715] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:43.558344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:43.570850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:43.577808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b98dc-f14a-459f-98be-69549a091241 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:43.620128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:43.620944 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.621248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.621548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.622306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:43.623861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:43.650172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 367/716] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:02:43 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 15:02:43.819083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6b472-6cbd-42fa-ab2d-509a8d5a00ee None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:43.869760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:43.870575 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:43.870853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:43.871129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] PATH_INFO: `/v3/services/network` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:43.871801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:43.873173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:43.886723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:43.887911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:43.888867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 350/717] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:43 2022] GET /identity/v3/services/network => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:43.894585 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:43.933018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:44.003733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a66b41-7899-4a12-9ead-5d4e0b4c950e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:44.040185 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:44.040918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:44.041201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:44.041507 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:44.042225 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] RBAC: Adding query filter params (name=network) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:44.042529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:44.043962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42598a66-f9e9-4ad3-b863-21ac0c71001d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:44.049023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 368/718] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:02:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:44.054581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:44.136209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:44.166694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c32e-b877-45d9-9087-4e5de617f1fd None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:44.214000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:44.214681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:44.214961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:44.215235 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:44.215957 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] RBAC: Adding query filter params (type=network) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:44.216266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:44.217728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b90ca5-3d59-42c8-9983-59b4f0407535 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:44.227373 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 351/719] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:02:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:45.257718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7079d-7d34-4937-b1f2-a8c94cbcb5a9 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.258303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7079d-7d34-4937-b1f2-a8c94cbcb5a9 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.258303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7079d-7d34-4937-b1f2-a8c94cbcb5a9 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.259918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 369/720] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:45.265645 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.265645 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.266150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.285531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:45.287023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:45.388001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 352/721] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:45.409383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d62a60-2ab0-425c-9c70-72d7c9374739 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.409997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d62a60-2ab0-425c-9c70-72d7c9374739 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.410486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d62a60-2ab0-425c-9c70-72d7c9374739 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.412081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 370/722] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:45.422247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:45.438676 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.439035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.439235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.474251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:45.475467 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:45.513720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:45.549606 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2672f49-38d9-4dd6-a1a3-870034538f64 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:45.596087 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:45.596900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.597177 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.597490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.599867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 371/723] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:45.608532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:45.627240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:45.628363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:45.629779 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 353/724] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:45.630579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:45.636295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:45.665062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:45.762477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d50b-cbfc-4616-9d37-2d70dd2607a3 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:45.797660 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf36e28-f8f0-4430-8bbf-c7c1f78dbeae None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:45.806631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:45.807309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.807587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.807865 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.808603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:45.810014 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:45.835872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 372/725] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:02:45 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:45.856790 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:45.893332 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:45.894082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:45.894452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:45.894885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:45.895762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:45.896083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:45.897622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:45.910794 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 354/726] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:45.915884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.048317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:46.067565 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cd4b9-55c2-4a3c-8c32-8ba6a178f4ae None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.116381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.117314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.117664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.118000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.134055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:46.135554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-6c556c83-78fb-4960-ab17-60c870f5dab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:46.158535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:46.159601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:46.160972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:46.160972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:46.160972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:46.160972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:46.162869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 373/727] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:46.168598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.254132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2591a-bf82-4f98-b057-977c70f52004 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.254460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2591a-bf82-4f98-b057-977c70f52004 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.254680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2591a-bf82-4f98-b057-977c70f52004 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.256219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 374/728] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:46.261640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.262683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.263195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.291453 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:46.293346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:46.295817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333c7bc-bebf-49fb-a0c7-65f803ff84fb None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.336297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.337227 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.337701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.338138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.339053 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:46.339515 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:46.341038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:46.353634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 355/729] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:46.362708 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.384739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 375/730] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 15:02:46.407805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.425041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:46.490461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72691850-16f7-4e71-84b9-4e7893cdc23b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.496374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:46.551277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a53c11-ee78-40cf-afac-9137023bbf1a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.559403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.565144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.565723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.566209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] PATH_INFO: `/v3/users/nova` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.575363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:46.577200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:46.592881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:46.594084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:46.595166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:46.598437 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 356/731] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:46.603641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.665359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.666203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.666485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.666760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.706208 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:46.721919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:46.722995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:46.724419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:46.725166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479c816-1d70-4fc2-92c8-fc3ba9a3d26a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.725300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 376/732] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:46.729728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:46.734319 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.768291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.769314 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.769775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.770215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.771246 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:46.771733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:46.773370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:46.818766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 357/733] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:46.819951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.893708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf210-1e7a-418e-ac8c-14e322a78fe4 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.923735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:46.929614 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.930285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.930570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.930849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.931609 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:46.931919 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:46.933335 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eed48-eef3-48db-b6ea-ca7212a29d2b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:46.938215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c1511-fcbc-46ef-add3-5ac6fd75210e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:46.942889 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 377/734] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:46.944333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67374f-f351-430f-8609-7176c0f05a25 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.944627 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67374f-f351-430f-8609-7176c0f05a25 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.944907 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67374f-f351-430f-8609-7176c0f05a25 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.946176 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 378/735] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:46.951107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67374f-f351-430f-8609-7176c0f05a25 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:46.977577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:46.978731 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:46.979206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:46.979659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:46.991087 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:46.992803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-309b8993-5c32-459c-9e9b-e869c7e324a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:47.012400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:47.013566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:47.014656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:47.014656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:47.014656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:47.014656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:47.021701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 358/736] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:47.023441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.023954 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.024438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.044537 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:47.046181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:47.058665 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.059931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67374f-f351-430f-8609-7176c0f05a25 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.094857 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.095752 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.095941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.096218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.107033 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.108699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f51981e9-cc81-430e-b7d4-92fb84275618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:47.125939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:47.127389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:47.128739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:47.128739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:47.128739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:47.128739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:47.129018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 359/737] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:47.129018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.139923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 379/738] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:47.139923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681eea77-dc52-4007-96ab-6e3a01f46f94 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.139923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681eea77-dc52-4007-96ab-6e3a01f46f94 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.139923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681eea77-dc52-4007-96ab-6e3a01f46f94 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.142689 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 380/739] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:47.142689 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681eea77-dc52-4007-96ab-6e3a01f46f94 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.193005 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.243946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c34b4b-2c21-4aea-b8e0-62210cd2212c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.253791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681eea77-dc52-4007-96ab-6e3a01f46f94 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.283943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.284629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.284916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.285192 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.285964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:47.286274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.287676 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.298478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.299150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.299437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.299963 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.300084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 360/740] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:47.300457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:47.300785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.302174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.302360 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd074c-b45d-4dfe-8617-1003b7df9b17 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.313722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.314334 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 381/741] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:47.315653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.315949 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.316275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.333100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:47.334316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:47.409441 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 382/742] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 15:02:47.410615 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.417921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b8b2f-47f1-414c-b45b-099103348ec8 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.449532 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.453619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.454609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.454962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.455333 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.466576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:47.474618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:47.475851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:47.476816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 361/743] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:47.477670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.529885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fe139-2813-4770-a912-246df7ad6ebd None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.553759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.565850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.566668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.566955 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.567231 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.574100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.575574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:47.589184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:47.590373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 383/744] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:47.591400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.598792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fa4d5-b972-4d27-8428-cbe4de4922c3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.636399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.637218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.637534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.638148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.646706 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.648158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:47.662672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:47.663742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 362/745] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:47.664809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.697089 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.704769 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a3cff-f8b0-47e6-b68a-0792857bbe2f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.739127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.739804 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.740091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.740364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.741099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.742521 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.756664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 384/746] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:02:47 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:47.762377 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.779947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a839-85b0-4566-9697-e0d38b409274 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.824210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:47.847640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.850028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.850544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.851024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.852027 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:47.852538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.854217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.872081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 363/747] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:47.872081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.903977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7537fa-9230-44b2-8962-704d7230892f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:47.956188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:47.958758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:47.959261 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:47.959785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:47.960788 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:47.961756 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:47.963258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:47.979529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 385/748] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:47.979529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:47.998699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc083b6-2521-49c0-ab2b-a66bbe5de35a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.002652 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:48.040619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.041336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.041886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.043363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.046431 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:48.046753 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.048202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.067157 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 364/749] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:48.068217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.113729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:48.125493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876718d-00a4-4701-8be0-f45c9f2d1e63 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.177030 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2696a7-3597-4350-b10e-9afd234ea6c3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.181127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.182258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.182750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.183230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.197781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.200075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.214751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.215580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.215866 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.216147 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/b764ac4c637749019a830d8cf3a22e09/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.218245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=b764ac4c637749019a830d8cf3a22e09, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.219814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:48.225527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:48.226715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:48.228163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:48.228163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:48.228163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:48.228163 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:48.229493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 386/750] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:48.230967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.242766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 365/751] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:48 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/b764ac4c637749019a830d8cf3a22e09/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:48.245181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.370365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:48.453494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151e170-1135-422e-8b75-4cc03105ee14 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.477933 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d53b0-15e8-4d65-a50f-a5403de26286 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.480306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:48.506160 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.507219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.507741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.508279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.516082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.517126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.517439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.517722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.518463 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:48.518767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.520140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b458-ecbe-427f-87aa-bc61082e3ddc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.532219 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 366/752] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:48.535808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1b824-8ac0-46f2-b6aa-720a75eb1519 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.535808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1b824-8ac0-46f2-b6aa-720a75eb1519 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.535808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1b824-8ac0-46f2-b6aa-720a75eb1519 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.535808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 367/753] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:48.539103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1b824-8ac0-46f2-b6aa-720a75eb1519 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.594635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.596396 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.654324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1b824-8ac0-46f2-b6aa-720a75eb1519 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.681097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:48.692360 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.693397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.693785 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.694166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.702637 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.704336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d6f6cfae-fa5e-41f7-a3cf-3f6744ee8b5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:48.719857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:48.721112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:02:48.722421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:48.728749 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 368/754] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:02:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:48.730237 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372262-249c-4282-b040-d55603f1081c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.730438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 387/755] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:02:48.730692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372262-249c-4282-b040-d55603f1081c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.731075 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372262-249c-4282-b040-d55603f1081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.731751 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.754148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-c9372262-249c-4282-b040-d55603f1081c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:48.755358 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-c9372262-249c-4282-b040-d55603f1081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:48.773544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:48.837273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 369/756] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:48.837273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9372262-249c-4282-b040-d55603f1081c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.874551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ade676-6848-4ff2-8f65-c383aed31d6d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.922489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:48.923324 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:48.923720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:48.924086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:48.925034 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:48.925484 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:48.927022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:48.945590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 388/757] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 15:02:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:48.946626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:48.963729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9372262-249c-4282-b040-d55603f1081c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:48.972298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.010698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.012576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.013083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.013579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.029734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.032004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.054591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ecd62-c2e0-40a8-a022-af0f0fb76bc9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:49.056505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:49.057507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:49.058521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:49.058521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:49.058521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:49.058521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:49.061614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 370/758] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:49.062580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.067396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.093901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.094947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.095336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.095711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.096469 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.097974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:49.112458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:49.113633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:49.114561 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:49.118914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 389/759] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 15:02:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:49.119835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.195157 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530510f-8516-4ef6-9b66-0c01a555d082 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.213665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.227745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da59d-be16-4b42-a28d-023e073cd226 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.243085 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.244136 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.244841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.245707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.273473 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.273782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.274161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.274335 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.274965 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:49.275369 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.280634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.294287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 390/760] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:49.295285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.345888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.348397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.352141 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.410434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ea63-99fb-4b70-92ba-f69c9a69eea1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.446143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.447280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.447280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.447472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.448091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:49.448399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.449834 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.459977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 391/761] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:49.460855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.475890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.494098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 371/762] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:02:49.495027 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.593526 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c8751-2c95-40cb-b047-4468c06729a9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.601759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b17d9-aa7a-42f7-81b7-6f1d9086bd30 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.605608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.644094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.644831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.645115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.645455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.646181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:49.646489 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.647264 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.647862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.648558 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.649387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.649691 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.659099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.660552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 372/763] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:49.661168 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.661469 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:49.677395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:49.678488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:49.679503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 392/764] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:49.680090 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.774830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14fb5c-33b5-4ba0-8608-66f18525933e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.776487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.811023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ad659-b67d-4e32-b677-ce68688cf0e0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.819010 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.820019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.820818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.821287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.835178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.837400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.849065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:49.849821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:49.850203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:49.850595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:49.851484 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] RBAC: Adding query filter params (interface=public, service_id=2058825aa7404a7486a64af4e0be0174, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:49.851898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:49.853417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:49.861059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:49.862316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:49.863408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:49.863408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:49.863408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:49.863408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:49.865264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 373/765] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:49.866290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.867643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 393/766] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/endpoints?service_id=2058825aa7404a7486a64af4e0be0174&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:49.868651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:49.894294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:49.987720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5a64a-1af6-4ebf-849b-878dc77fdbcd None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:49.995253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f38be4-1a4c-44e1-8877-5e9e8a23df5e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.025290 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.026238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.026683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.027143 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.028119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.030009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.032634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:50.034930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.035778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.036209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.036656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.037357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 394/767] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:50.045408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.046262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:50.046744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.049319 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.063711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 374/768] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:50.070589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.186576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:50.190281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4764d-bca4-4d8d-b7e5-f151d5d9e717 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.212131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6179a8-12ef-40a1-8a31-a7546bf68010 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.229851 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03472203-466f-4ebe-9536-d25192374835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.230516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03472203-466f-4ebe-9536-d25192374835 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.230799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03472203-466f-4ebe-9536-d25192374835 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.231079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03472203-466f-4ebe-9536-d25192374835 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.231869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03472203-466f-4ebe-9536-d25192374835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:50.232126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03472203-466f-4ebe-9536-d25192374835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.233663 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03472203-466f-4ebe-9536-d25192374835 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.247052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 395/769] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:50.248587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95e52-22e6-4141-b57c-d51ec5869009 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.248873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95e52-22e6-4141-b57c-d51ec5869009 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.249149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95e52-22e6-4141-b57c-d51ec5869009 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.250799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 396/770] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:50.252207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95e52-22e6-4141-b57c-d51ec5869009 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.257151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.258160 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.258685 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.259174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.260287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=6f8c56047dbd40928ac7cb29bbc767ba) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:50.260827 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.262677 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256963b1-fd08-460f-b9ac-6472d30216d9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.275095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 375/771] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/role_assignments?user.id=6f8c56047dbd40928ac7cb29bbc767ba&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:50.276532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.276781 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.277064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.312289 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:50.315291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:50.360746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:50.410764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95e52-22e6-4141-b57c-d51ec5869009 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.443421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 376/772] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:50.462846 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.463703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.463985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.464266 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.479715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.480269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.481255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:50.496057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:50.497159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:50.498216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 397/773] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:50.506548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.623121 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae501509-9727-4c8d-b720-51373adf141c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.646451 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:50.663874 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.664926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.665454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.665971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.667105 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:50.667649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.669417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.675351 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:50.686280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 398/774] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:50.694965 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.732309 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b9b92-a551-429d-98f4-5459621e8a4c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.792847 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.793701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.793989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.794312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] PATH_INFO: `/v3/services/network` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.794998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.796535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-99b7d8ed-82bc-49ac-98f2-f38a2f198191 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:50.808604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:50.809951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:50.811148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 377/775] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/services/network => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:50.812257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979e2c-4b30-4cfc-9443-e1474f124925 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.812257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979e2c-4b30-4cfc-9443-e1474f124925 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.812257 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979e2c-4b30-4cfc-9443-e1474f124925 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.813358 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 378/776] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:50.818364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.818740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.819092 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.836690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:50.838039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:50.852713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07ccd-7bd8-4ec3-be64-f2f47deaf92b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:50.889735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:50.890839 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:50.891305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:50.892215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:50.893349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=daf0d21e592f4777becaaf853c52e043) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:50.893877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:50.895642 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:50.902348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 399/777] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/role_assignments?user.id=daf0d21e592f4777becaaf853c52e043&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:50.903410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.925275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 379/778] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:02:50.960773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:50.985222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:51.095566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cbf3c-64fc-4e84-9580-2975103bfc57 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.099186 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:51.138063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.138745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.139118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.139398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.140130 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] RBAC: Adding query filter params (name=network) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:51.140445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:51.142556 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:51.150611 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 400/779] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:51.155992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:51.170291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecac473-5b96-4cdb-a551-1a0e2c656eed None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.215160 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.216369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.216855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.217403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.238465 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:51.250898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:51.251947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:51.252892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:51.258481 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 380/780] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:51.264542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:51.329165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1d0e-1d82-4ea3-afef-9763c99cf37c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.349518 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:51.387241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a156307-0ba9-456d-900a-d7b831c1df93 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.417689 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:51.426057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.427354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.429169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.430021 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.430527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.431449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] RBAC: Adding query filter params (type=network) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:51.432337 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.432833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.433363 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:51.433834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.435458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:51.436649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:51.437201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:51.440268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:51.449571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 401/781] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:02:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:51.453089 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:51.463255 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 381/782] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:51.464818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:51.591335 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87611a-298a-4824-b468-ba5f25b0d8b0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.632650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.633792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.634282 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.634784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.635798 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] RBAC: Adding query filter params (interface=public, service_id=98e2783bb2724cd78836b6ae41b4b42f, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:51.636314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:51.637940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:51.645417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 402/783] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:02:51 2022] GET /identity/v3/endpoints?service_id=98e2783bb2724cd78836b6ae41b4b42f&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:51.650488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:51.674380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32881497-db45-4595-92a5-3fa093221e0a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.696733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:51.746478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.748106 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.749184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.752057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.773135 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:51.774858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:51.797977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:51.799222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:51.800441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:51.800441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:51.800441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:51.800441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:51.802558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 382/784] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:51.808165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:51.833916 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329ca47-6740-470d-8642-757979a81b59 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:51.892904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:51.906156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:51.907065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:51.911522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:51.911983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:51.912853 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:51.914479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:51.923382 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 403/785] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:02:51 2022] GET /identity/v3/services => generated 772 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:51.981687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85688cd6-1b20-49a7-86e9-ad7066c8422f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:52.037141 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:52.038194 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.038667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.039107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.040095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:52.040578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:52.042234 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:52.063970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 383/786] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:52.070186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:52.230734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:52.233977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b849194-68c5-4382-823c-7a2004b34202 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:52.280775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:52.280775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.280775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.280775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.290935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:52.292427 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:52.312354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:52.313512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:52.314616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 404/787] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:02:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:52.319456 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:52.481348 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708a189-7c33-4508-be02-70c171d27aca None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:52.483205 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:52.533148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:52.533848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.534133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.534404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.535256 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:52.535560 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:52.536992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a162e9-4612-4a91-a806-829832f686cc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:52.561004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 384/788] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:02:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:52.566599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:52.706249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344389a-25c2-4743-9cc9-7e6215e8dd8c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:52.757255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:52.757838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.758120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.758396 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.779000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776537cf-487e-4a42-a597-785a99dfa479 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.779736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776537cf-487e-4a42-a597-785a99dfa479 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.779832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776537cf-487e-4a42-a597-785a99dfa479 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.781510 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 385/789] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:52.787059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72757bf-5755-41ff-b11b-e5fc09b1671b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.787349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72757bf-5755-41ff-b11b-e5fc09b1671b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.787629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72757bf-5755-41ff-b11b-e5fc09b1671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.797586 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:52.806275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-b72757bf-5755-41ff-b11b-e5fc09b1671b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:52.807531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-b72757bf-5755-41ff-b11b-e5fc09b1671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:52.864628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:52.866234 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:52.932094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 386/790] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:52.933649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215b37f-0ee5-4a72-a41f-c7815edd384e None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.933939 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215b37f-0ee5-4a72-a41f-c7815edd384e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.934219 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215b37f-0ee5-4a72-a41f-c7815edd384e None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.939847 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 387/791] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:52.941615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2309b-3752-4d53-a128-228041be533c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:52.941903 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2309b-3752-4d53-a128-228041be533c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:52.942186 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2309b-3752-4d53-a128-228041be533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:52.962382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-a1a2309b-3752-4d53-a128-228041be533c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:52.963588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-a1a2309b-3752-4d53-a128-228041be533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:52.995789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 405/792] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:02:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:02:52.996679 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.047092 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 388/793] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:53.047951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2309b-3752-4d53-a128-228041be533c None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.059474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.077346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.104735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b946-a2ab-4c9f-8199-25f20b2a6529 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.139976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.140804 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.141085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.141408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.142443 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.143845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:53.156166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:53.157284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:53.158273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 406/794] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 19 15:02:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:53.159248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.167169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2309b-3752-4d53-a128-228041be533c None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.205053 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.205904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.206283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.206574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.217825 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.219297 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:53.238499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:53.240668 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:53.243011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:53.243011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:53.243011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:53.243011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:53.245293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 389/795] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:53.246080 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.264517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f27117-d791-4f47-a11d-0ecdfed64c84 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.288515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.299827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.300891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.300891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.301203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.312271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:53.320029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:53.321317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:53.322503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:53.326706 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 407/796] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:53.327132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.330872 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.367053 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d336d-e119-4a0a-9738-b1453035778d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.408827 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.409809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.410280 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.410754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.411682 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:53.412178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.413834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.419701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 390/797] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:53.420753 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.444016 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e370d0-bacc-4f66-8cd9-49fd1aae816c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.487553 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.488290 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.488574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.488895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.489658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:53.489969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.491465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.509202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 408/798] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:53.509202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.559550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.561766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87651e1d-9210-4543-b3ac-04237be90c27 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.604960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.606738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.607403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.608011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.609226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:53.609705 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.611117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.613685 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.615491 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d1c94-d1c5-4cdc-9170-c46826c5c838 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.623388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 391/799] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:53.626584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.652008 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.652685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.653056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.653274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.653977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:53.654489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.655740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72569d9c-d585-4d5a-9dd1-03f4f9077c54 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.665106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 409/800] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:53.665106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df903ea9-5657-4d0a-92db-c0c2825ac3a7 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.665106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df903ea9-5657-4d0a-92db-c0c2825ac3a7 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.665106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df903ea9-5657-4d0a-92db-c0c2825ac3a7 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.665106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 410/801] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:53.665106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df903ea9-5657-4d0a-92db-c0c2825ac3a7 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.737158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9b77-dec7-4f74-ac9b-e3aa8e0d0b6d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.772716 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df903ea9-5657-4d0a-92db-c0c2825ac3a7 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.776370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.777183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.777493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.777771 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.783715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.785254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.786998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:53.802201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:53.803355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:53.804416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 392/802] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:53.804949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.816607 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.817404 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.817680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.817962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.828263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.829764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-d05c7064-cc5a-47cd-a208-0146e308f843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:53.848409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:53.849476 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 411/803] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.855383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.870745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:53.871933 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:53.872733 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:53.922808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e7bf-a48b-4c76-a5fc-4b42531dadb3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:53.953522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 412/804] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:53.954329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:53.958735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:53.959422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:53.959701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:53.959988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:53.960741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:53.962888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:53.985497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 393/805] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:02:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:53.986997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.085804 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:54.099128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68a034-92b7-410e-9e93-707f43c9e940 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.135894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74f749-473b-410c-8e27-4792c26dae9a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.145579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.146903 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.147198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.147480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.148320 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:54.148634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:54.150666 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:54.152137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:54.169434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 413/806] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:54.170910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.187730 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.188801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.189385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.189966 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.191024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:54.191560 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:54.193160 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:54.211291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 394/807] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:54.211291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.315692 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee2bf8-c69d-42b5-b3dc-c56853d4b18d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.335424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af196a-58c1-422f-9223-9ade41089b42 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.352330 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.353293 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.353587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.353866 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.365569 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:54.374180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:54.375302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:54.376253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:54.380715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 414/808] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:54.381909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.384321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.385406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.386269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.387889 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.389314 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=b764ac4c637749019a830d8cf3a22e09) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:54.389856 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:54.391610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:54.400070 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:54.408768 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 395/809] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/role_assignments?user.id=b764ac4c637749019a830d8cf3a22e09&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:54.410572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.512408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:54.528851 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a981f8-ced8-4595-8cf2-0e1a92f0f66c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.588097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.588926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.589203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.589510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] PATH_INFO: `/v3/users/swift` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.602134 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:54.605365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-e6eea32b-2d95-4064-b569-23894e673fbc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:54.624050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:54.625659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:54.626759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 415/810] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:54.627962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740075e-3cc1-480d-b0d6-0c2243f31a70 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.627962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740075e-3cc1-480d-b0d6-0c2243f31a70 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.628149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740075e-3cc1-480d-b0d6-0c2243f31a70 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.630393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 416/811] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:54.633311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740075e-3cc1-480d-b0d6-0c2243f31a70 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.649026 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6da7f-fc3c-47a3-ab7b-4c8fcd089f45 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.689119 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.689840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.690137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.690402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.691175 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:54.691483 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:54.692924 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7beaad-d6a8-4813-ab8c-1626e317ea05 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:54.701878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 396/812] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:54.703284 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.703579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.703860 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.720212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:54.720520 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:54.721781 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:54.764713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740075e-3cc1-480d-b0d6-0c2243f31a70 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.801998 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.802681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.802967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.803243 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.804087 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:54.804410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:54.805885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:54.806726 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 397/813] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:54.807903 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.841735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:54.847160 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 417/814] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:54.848332 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:54.924992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23a3ca-8c5c-41e8-90e5-a4b11b6fa7bd None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.979128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e893bec-7af5-40cb-8975-4c2ba89b5de3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:54.982824 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:54.983641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:54.983922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:54.984202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:54.998566 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.001080 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.010462 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.019999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.020814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.021096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.021402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] PATH_INFO: `/v3/services/network` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.022066 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.023457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:55.029002 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:55.030137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:55.031153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:55.031153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:55.031153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:55.031153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:55.037600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:55.038729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 15:02:55.039685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:55.040541 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 398/816] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:55.040541 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.041921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 418/816] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:54 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:55.042796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.134459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.159478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4287fc-3855-4912-89e8-0836164dd003 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.175618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72840ef-e76b-4772-99d7-82c66e4244ee None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.209993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.210659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.210943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.211218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.211949 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:55.212251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.212353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.213307 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.213664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.213764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.214019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.226909 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.228480 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.229595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 419/817] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:55.230496 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:55.251199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:55.252295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:55.253343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:55.253343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:55.253343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:55.253343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:55.253343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 399/818] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:55.255217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.301713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.341119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c0c8-a971-4253-85ca-6f3773bae57a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.382839 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.384222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.384957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.385787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.386750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] RBAC: Adding query filter params (name=network) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:55.387260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.388887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db603-5941-4093-b2de-cbcbf62ef2ee None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.389935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.396415 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 420/819] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:55.397602 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.427631 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.438912 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.439754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.440041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.440316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.450545 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.452089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:02:55.470612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:55.471686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:02:55.472746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:55.479989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 400/820] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:55.481147 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.509063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581fd35-c3dd-46d2-ba3d-dad71384c21a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.554938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.555617 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.556521 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.556521 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.557123 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:55.557542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.559349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.578849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 421/821] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:55.578849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.608897 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e376e-8c55-4d18-a7dc-c4437369280f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.633743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.651065 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.652164 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.652760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.653365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.654296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] RBAC: Adding query filter params (type=network) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:55.654916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.656647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.662711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 401/822] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:55.663841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.701417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1ebcf-b633-4f11-8ec8-c1c8aa9ec03e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.733386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.738948 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.739952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.740477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.740968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.742071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:55.742620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.744297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.765438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 422/823] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:55.767200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.783202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb864df-3fb6-402e-a5c4-a49866e2e40c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.810110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:55.823833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.824645 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.824926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.825198 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.833067 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.834557 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:55.849943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:55.851158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:55.852202 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 402/824] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:55.852756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:55.912420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6866b3-b69a-4821-a188-ece766b2b6e3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:55.969103 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:55.970025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:55.970309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:55.970645 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:55.971465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:55.973031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.000738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280a03b-d6df-4474-aacf-be7d8a615f85 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.005535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 423/825] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:02:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:02:56.007828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.034176 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:56.089909 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.091064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.091440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.091826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.112527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.116662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:56.148104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:56.149478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:56.150699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:56.150699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:56.150699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:56.150699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:56.152890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 403/826] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:56.159609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.204620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:56.225028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a152ad0-0ba0-4db3-b455-261edbe59ba7 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.285782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.286763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.287203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.287695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.288504 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:56.289806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.291394 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.307331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 424/827] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:56.312921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.362715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ec8bc-692a-45b3-adfb-871c714a9e0b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.405354 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.406203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.406596 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.406993 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.407839 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:56.408253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.409849 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.425387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 404/828] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:56.432562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.464915 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9b1c8-ac9b-4ef2-998d-86a992cc8a11 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.473639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:56.505613 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26580169-d537-478f-b930-920862a6c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.506436 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26580169-d537-478f-b930-920862a6c3ea None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.506721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26580169-d537-478f-b930-920862a6c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.506995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26580169-d537-478f-b930-920862a6c3ea None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/6f8c56047dbd40928ac7cb29bbc767ba/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.508992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26580169-d537-478f-b930-920862a6c3ea None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=6f8c56047dbd40928ac7cb29bbc767ba, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.510495 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26580169-d537-478f-b930-920862a6c3ea None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.531808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 425/829] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:56 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/6f8c56047dbd40928ac7cb29bbc767ba/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:02:56.531808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ca39-877c-4e34-8f3f-2b963c77d33c None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.531808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ca39-877c-4e34-8f3f-2b963c77d33c None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.531808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ca39-877c-4e34-8f3f-2b963c77d33c None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.531808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 426/830] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:56.546872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.547259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.547546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.564807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b21e4-4cdf-4cb0-91c9-b1d9af6e78f2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.566785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:56.568117 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:56.573258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:56.615449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.616303 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.616585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.616862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.625270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.626817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:02:56.642654 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:56.643772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:02:56.644808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:56.648923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 405/831] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:02:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:56.650263 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.665937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 427/832] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:02:56.689503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.772757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:56.796412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949316c4-9a00-46cc-b1fb-8a7297f5645a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.819761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcc7d7-4e36-4151-af43-079be0f727b3 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:56.832976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.834424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.834975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.835470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.836550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:56.837074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.838758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.854906 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 406/833] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:56.861291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.867044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:56.868326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:56.868326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:56.868605 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:56.869265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:56.870678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:56.884141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:56.885352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:56.886326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 428/834] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:56.893727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:56.898079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:56.973324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33481dc7-a982-4fc3-9da9-5f1f7778a2ba None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:57.006349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1052cc-0421-4e3a-9ecf-72c0e0f9dde2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:57.010806 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8019343-0b14-47ba-bdfa-062ca9a01cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:57.011715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8019343-0b14-47ba-bdfa-062ca9a01cb2 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.011989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8019343-0b14-47ba-bdfa-062ca9a01cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.012282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8019343-0b14-47ba-bdfa-062ca9a01cb2 None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/daf0d21e592f4777becaaf853c52e043/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.025061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:57.028321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8019343-0b14-47ba-bdfa-062ca9a01cb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=daf0d21e592f4777becaaf853c52e043, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:57.030007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8019343-0b14-47ba-bdfa-062ca9a01cb2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:57.044018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:57.044701 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.045110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.045360 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.053192 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 407/835] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:02:56 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/daf0d21e592f4777becaaf853c52e043/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 15:02:57.053431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:57.053431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:57.053638 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c35d7-3d9d-473d-863d-cdfe96b6aa72 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.053638 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c35d7-3d9d-473d-863d-cdfe96b6aa72 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.053638 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c35d7-3d9d-473d-863d-cdfe96b6aa72 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.053913 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:57.054004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 408/836] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:57.054780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.055248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.055377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.059766 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 429/837] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:57.065367 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:57.080011 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:57.081295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:57.165218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 409/838] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:57.170758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:57.192821 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:57.208895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0bdd0-97fe-4c88-9764-cefc3dc153d3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:57.256014 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:57.256705 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.256992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.257303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.258038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:57.258351 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:57.259758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:57.267477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 430/839] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:02:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:57.338313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e971f-148f-43a7-9baf-89c73138c428 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:57.365660 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:57.378838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:57.379748 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.380034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.380328 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.381036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:57.382519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:57.397029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:57.398200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:57.399162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 410/840] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 15:02:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:57.406397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:57.520187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278c33d-bfc0-4a81-bfcc-22cbb8a50c7e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:57.549706 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:57.555758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:57.556446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.556731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.557070 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.557763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:57.558238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:57.559833 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeecc6d-d596-45d1-994d-c9beb38a32f6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:57.569434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 431/841] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:02:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:57.570802 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:57.678191 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749a8f5-c574-4c7a-8197-69ee0c24827b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:57.717476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:57.718162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:57.718448 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:57.718726 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:57.719450 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:57.719761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:57.721145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79989fc-712e-4944-8457-da4cf9562675 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:57.726225 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 411/842] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:02:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:58.892910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffc107-4685-4194-9c82-012465f79a46 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:58.893543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffc107-4685-4194-9c82-012465f79a46 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:58.893543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffc107-4685-4194-9c82-012465f79a46 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:58.895169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 432/843] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:58.901221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:58.901833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:58.901833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:58.919729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:58.920971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:58.969704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:58.997369 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 412/844] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:59.015999 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffc107-4685-4194-9c82-012465f79a46 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.126211 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffc107-4685-4194-9c82-012465f79a46 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.161997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:59.163190 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.163691 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.164296 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.182275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:59.191995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:59.193361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:02:59.194664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:59.197762 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 433/845] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:59.209284 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.309528 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee15733-82da-41e9-91b9-ca0af26dc143 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.346017 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:59.347117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.347622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.348116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.349106 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:59.349780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:59.351396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63168b14-6044-4936-8d2f-3d40831aafb7 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:59.365338 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 413/846] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:59.367297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.388210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403d9e-6024-4049-ba23-05ba75343751 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.388545 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403d9e-6024-4049-ba23-05ba75343751 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.388846 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403d9e-6024-4049-ba23-05ba75343751 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.390536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 414/847] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:02:59.396129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.396439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.396702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.417709 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:59.419467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:59.432383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:59.465653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:59.487288 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694d21c-3ae1-4a16-b441-53bce0f0669e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.507251 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 415/848] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:02:59.524500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:59.525351 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.525691 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.525967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.528636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.537547 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:59.539147 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:02:59.558786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:59.559931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:59.561136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:02:59.561136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:02:59.561136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:02:59.561136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:02:59.564980 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 434/849] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:02:59.566232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.640283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8f7f3-ca3c-4d6a-bd9f-586803e21587 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.674624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8940a38-448a-4c1d-9054-7400e9d7b0ce None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.677511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1449a58-2bb7-4bb1-bed5-4b6dbebe7a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:59.677711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449a58-2bb7-4bb1-bed5-4b6dbebe7a7c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.677981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449a58-2bb7-4bb1-bed5-4b6dbebe7a7c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.678250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449a58-2bb7-4bb1-bed5-4b6dbebe7a7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.684792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:59.690303 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1449a58-2bb7-4bb1-bed5-4b6dbebe7a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-a1449a58-2bb7-4bb1-bed5-4b6dbebe7a7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:02:59.698636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:02:59.699783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:02:59.700807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 416/850] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:02:59.702326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297647c-1060-4e51-a7ff-b79f0f65b929 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.702326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297647c-1060-4e51-a7ff-b79f0f65b929 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.702326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297647c-1060-4e51-a7ff-b79f0f65b929 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.707393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 417/851] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:02:59.707393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297647c-1060-4e51-a7ff-b79f0f65b929 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.723095 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:59.723785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.724070 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.724346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.725080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:59.725432 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:59.726827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425765bb-c2f9-45a1-9cee-04e0b9064846 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:59.743318 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 435/852] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:02:59.743318 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.743318 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.743318 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.745446 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:59.757250 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:02:59.758588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:02:59.825761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297647c-1060-4e51-a7ff-b79f0f65b929 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.831706 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 436/853] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:02:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:02:59.842530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.873090 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:02:59.873924 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:02:59.874215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:02:59.874496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:02:59.875279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:02:59.875596 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:02:59.877075 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:02:59.887992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 418/854] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:02:59.889045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:02:59.963885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:59.968633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d9e3b-2026-486a-b244-068a5f2df1f0 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:02:59.987410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:02:59.995211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126623a2-7214-4de6-9898-f67ac5da795e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.007973 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.008880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.009184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.009482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.017811 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.019282 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.031452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.032122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.032409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.032687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.035302 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:00.036255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:00.037392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 15:03:00.038590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:00.039294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.042449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 437/855] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:02:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:00.043288 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.054269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 419/856] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:02:59 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:00.055329 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.177564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98347472-d2d5-4af5-b202-4c3a03fb0448 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.188816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367eb5a-8530-4f6a-af36-f6e5e855545e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.215881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.216621 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.217142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.217406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.228988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.229725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.229855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.230057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.230286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.231151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:00.231265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.231462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.233036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174944f-6efa-4390-b4ed-ca74ec4d6a4f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-b537e004-022b-4e0f-9e01-cb236725e55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:00.250220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:00.251362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:00.252437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:00.252437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:00.252437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:00.252437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:00.252437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 438/857] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:00.252988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eeaa1-3f2c-40ec-96ce-fca24c8b3a23 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.253227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eeaa1-3f2c-40ec-96ce-fca24c8b3a23 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.253449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eeaa1-3f2c-40ec-96ce-fca24c8b3a23 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.254728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 439/858] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:00.258340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 420/859] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:00.259281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eeaa1-3f2c-40ec-96ce-fca24c8b3a23 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.262411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.262411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.262411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.282201 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:00.289588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:00.290807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:00.293060 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:00.373339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 421/860] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:00.373339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.376651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eeaa1-3f2c-40ec-96ce-fca24c8b3a23 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.413835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.414514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.414800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.415076 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.415815 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:00.416115 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.417550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.429205 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 440/861] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:00.430085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.484994 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf247084-6061-4ce4-b289-f0ac99b44e15 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.507131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:00.524749 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.525438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.525723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.526007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.535634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:00.543558 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a131-706a-42c8-bf91-9cc4081844df None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.577937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.578812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.579006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.579272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.580109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.581467 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.597137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 441/862] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:03:00 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:00.597905 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.615438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.617900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.730250 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5099b-7767-4bbd-80a5-6971c82c7c5e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.763408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 422/863] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:00.770797 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.775529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:00.787409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.789075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.789977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.790544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.800046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.802153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.818062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:00.820237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:00.821430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:00.822616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 442/864] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:00.827723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.900608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d2d36-e07b-44c6-a953-b2843c97e83a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.935582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.936398 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.936682 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.936961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:00.947640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:00.949102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:00.953639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace3d4-15ec-4702-aa18-2bb484e4ccd7 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:00.967247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:00.968384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:00.969513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:00.969513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:00.969513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:00.969513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:00.969513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 423/865] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:00.974340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:00.998336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:00.999079 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:00.999333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:00.999577 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.000431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:01.000739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.002236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.025873 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 443/866] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:01.026800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.081709 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.093190 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46eea9-b3a8-41d1-a7d7-89b84c42c794 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.097708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.131058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39256-314d-48ef-9d27-46f5edd01d7b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.134407 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.135091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.135387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.135666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.136414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:01.136728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.138243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.151111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 424/867] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:01.157201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.179075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.180428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.180965 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.181511 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.274766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294df51-5894-4530-8f02-3f6310fce5d1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.277951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.279477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.315575 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.316782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.317336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.317872 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.325574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.326098 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.328608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:01.347122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:01.348269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:01.349385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:01.355970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 425/868] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:01.357078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.370692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.407588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 444/869] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 15:03:01.408710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.486095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb7980-a3a0-4695-9fc8-c6cbdcb45ffb None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.510157 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4971c-3a96-485e-a3fa-9e89da09790c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.532288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.533629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.534310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.534808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.536404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:01.536958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.539361 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.545986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.546799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.547080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.547356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.560716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 426/870] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:01.562460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.563926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.566845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.569806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:01.592075 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:01.593390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:01.594510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:01.594510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:01.594510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:01.594510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:01.597392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 445/871] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:01.600114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.646826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.682102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158f0b-2c87-469d-9203-27a52955bd65 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.719244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.720196 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.720649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.721158 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.722247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=6f8c56047dbd40928ac7cb29bbc767ba) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:01.722791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.724449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.731069 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 427/872] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/role_assignments?user.id=6f8c56047dbd40928ac7cb29bbc767ba&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:01.744730 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28ab6b-4bc1-48a3-a170-7fa9af96882b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.786364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.787416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.787911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.788388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.789393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:01.789929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.791574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:01.805576 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 446/873] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:01.813895 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:01.942080 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72143af5-5192-4aca-a230-2d8608599094 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:01.981746 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:01.982977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:01.983270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:01.983554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:01.985672 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:01.991973 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:01.993535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:02.009598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:02.010719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:02.012475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 428/874] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:02.017806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:02.084753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:02.135125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94cd43-7a26-46d8-99e4-fdb7f571355e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:02.176806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:02.177537 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.177827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.178112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.178943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:02.179253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:02.180758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080c885-325d-4503-b286-5252e79c2e23 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:02.194827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 447/875] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:02.201034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:02.310744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad082d0-dc33-4437-baf8-09586115cf7c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:02.348113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:02.349278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.349278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.349278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.351139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=daf0d21e592f4777becaaf853c52e043) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:02.351139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:02.351916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cdeb1-9d97-4609-8603-768a6c28949b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:02.359151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 429/876] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:02 2022] GET /identity/v3/role_assignments?user.id=daf0d21e592f4777becaaf853c52e043&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:02.368021 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c850ea-7761-4ba8-8481-049c85dc8ec1 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.368381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c850ea-7761-4ba8-8481-049c85dc8ec1 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.368573 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c850ea-7761-4ba8-8481-049c85dc8ec1 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.370385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 448/877] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:02.376452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.376830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.377079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.396298 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:02.397577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:02.463707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:02.479107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 430/878] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:02.503142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c850ea-7761-4ba8-8481-049c85dc8ec1 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:02.617745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c850ea-7761-4ba8-8481-049c85dc8ec1 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:02.635576 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:02.654775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:02.655656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.655941 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.656220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.656891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:02.658413 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:02.671387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:02.672709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:02.674387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 449/879] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:03:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:02.679180 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:02.791852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76a62d-5893-4201-b01c-ab2685f91045 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:02.830775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:02.831840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.832331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.832811 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.833789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:02.834306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:02.835885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13078538-5669-462f-ba87-49e2456846e7 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:02.841679 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 431/880] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:03:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:02.847856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:02.952760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae215f94-b651-4810-a3ef-1cf71968c2ec None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:02.962880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde9e61-d848-47c1-a928-4e480badd59c None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.963421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde9e61-d848-47c1-a928-4e480badd59c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.963935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde9e61-d848-47c1-a928-4e480badd59c None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.965868 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 432/881] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:03:02.971052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.971597 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.972106 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.990656 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:02.992323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:02.993829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:02.994514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:02.994803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:02.995088 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:02.995818 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:02.996132 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:02.998967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.013009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 450/882] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:03:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:03.020353 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.057793 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:03.078147 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:03.088845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 433/883] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:03.104581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.154971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b4e83-fd15-45e5-b3b3-f0b5d5f24733 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.200993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.202301 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.203253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.203834 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.204904 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] RBAC: Adding query filter params (interface=public, service_id=230dd75890134765bb1a7d472bb5d32d, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:03.205646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.207854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.208887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4d4c-c4c7-4d61-9ca8-0f4717ad44bf None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.215579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 451/884] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/endpoints?service_id=230dd75890134765bb1a7d472bb5d32d&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:03.221397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.247118 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.248034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.248323 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.248602 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.249362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.250714 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:03.263547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:03.264762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:03:03.266272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:03.271928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 434/885] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:03.271928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.321860 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:03.344341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a035bb-6bf0-4857-a5e1-d7eea9b942eb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.354094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:03.379618 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb292cad-900e-4bdb-9cfa-c4c45be89c34 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.381428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.382319 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.382775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.383215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.384058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.385727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.401399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 452/886] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 15:03:03.418108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.418896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.419184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.419504 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.420319 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:03.420636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.422127 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.428729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 435/887] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:03.434650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.545863 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eacb84-c05d-4e52-aa6d-47cd823e5b32 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.586389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.587422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.587878 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.588340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.589289 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:03.589770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.591388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.598136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 453/888] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:03.606716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.722796 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a08b4-47a7-46f4-906c-bd4f3e57adbb None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.734073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:03.759270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.760238 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.760692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.761138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.762139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] RBAC: Adding query filter params (interface=public, service_id=785dfef935454ba7a2f2ca67e71e40bc, region_id=RegionOne) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:03.762639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.764260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bec3e-42db-496f-972b-4beccff7dc73 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.773381 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 436/889] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/endpoints?service_id=785dfef935454ba7a2f2ca67e71e40bc&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:03.778464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:03.816785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:03.886510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c79ddd-231b-46e2-986e-6a917e87e419 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:03.921680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbda439-8465-4c36-8e62-9c21118c8802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:03.922678 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbda439-8465-4c36-8e62-9c21118c8802 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:03.923135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbda439-8465-4c36-8e62-9c21118c8802 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:03.923591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbda439-8465-4c36-8e62-9c21118c8802 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:03.924443 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbda439-8465-4c36-8e62-9c21118c8802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:03.926026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbda439-8465-4c36-8e62-9c21118c8802 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:03.937325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 454/890] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:03:03 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 15:03:04.166633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acc696-44fc-40bb-9c4a-336ea73d6a96 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.167025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acc696-44fc-40bb-9c4a-336ea73d6a96 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.167215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acc696-44fc-40bb-9c4a-336ea73d6a96 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.168767 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 437/891] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:04.174674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.175587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.176114 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.194165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:04.195764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:04.267718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 455/892] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:04.291291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acc696-44fc-40bb-9c4a-336ea73d6a96 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:04.404093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acc696-44fc-40bb-9c4a-336ea73d6a96 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:04.439420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:04.440215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.440505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.440787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.441491 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:04.442952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:04.457273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:04.458525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:04.459572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 438/893] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:04.467434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:04.505647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:04.574606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671a057-0057-4b50-b87d-e470b3f1c59f None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:04.614259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:04.614946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.615233 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.615515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.616242 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:04.616553 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:04.617991 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec96da-f895-4301-ad68-4a72617b7ba7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:04.623928 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 456/894] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:04.630096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:04.677608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:04.736680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77a670-c883-4b86-8ee9-3bf7fd9c27e2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:04.769113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a3423-bf3d-491d-b7b2-b7fe083cb8d2 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.769713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a3423-bf3d-491d-b7b2-b7fe083cb8d2 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.770158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a3423-bf3d-491d-b7b2-b7fe083cb8d2 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.771940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 457/895] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:04.775862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:04.775862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.775862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.775862 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.776293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:04.776293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:04.776293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:04.777279 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:04.777567 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:04.777845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:04.781276 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 439/896] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:04.801939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:04.803557 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:04.880819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 458/897] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:04.904924 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.019871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dbe71-397a-4821-a363-283f8a8664ec None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:05.021888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:05.057038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:05.057913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.058199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.058475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.069457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:05.077721 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:05.078832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:05.080438 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 440/898] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 15:03:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:05.085361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.110562 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:05.190148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcb473-f567-430a-8a31-20146c39f43d None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:05.228375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:05.229410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.229866 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.230301 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.231237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:05.231718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:05.233374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:05.242689 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 459/899] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:05.248165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.369012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d83024-29f5-4fad-b3b7-5c54ef6660fe None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:05.404783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:05.405681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.405970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.406249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.417440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:05.418970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:05.436778 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:05.437910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:05.438988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:05.438988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:05.438988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:05.438988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:05.438988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 441/900] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:05.446393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.533516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:05.548721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e544dec-affb-4b7b-a3f4-af7deb34506d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:05.583596 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:05.584300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.584591 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.584867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.585658 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:05.585969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:05.587461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3d0c-5fa5-4485-b88d-9ecad581fc7f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:05.604992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 460/901] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:05.610612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.627838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:05.717527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee97b6-3669-4733-b123-6080073c4ce8 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:05.752729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:05.753599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.753886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.754165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.762193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:05.763723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:05.778900 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff7185-4b3d-434a-bf46-0ed9ddfc0e6a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:05.779478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:05.780584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:05.781775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 442/902] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:05.783141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff7185-4b3d-434a-bf46-0ed9ddfc0e6a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.783713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff7185-4b3d-434a-bf46-0ed9ddfc0e6a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.785754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 461/903] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:05.789722 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.791408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.791951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.792451 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.812293 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:05.814361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:05.869826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:05.902045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a585489-61be-4d24-9eaf-67ea9290df31 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:05.903927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 462/904] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:05.926937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:05.947097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:05.947782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:05.948067 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:05.948346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:05.949197 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:05.949747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:05.951502 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:05.970408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 443/905] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:05.976337 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.013204 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.040196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050723b-767c-4094-baaf-06f3a8041ede None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.076799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.078069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.078604 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.079090 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.079981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.081657 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.087970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce10a96-ae2d-43b7-b083-267be85a54e7 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:06.095895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:06.097119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 15:03:06.098199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:06.106119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 463/906] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:06.107412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.121147 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.128048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.128719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.129004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.129392 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.211270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.212834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.218274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4bb62-f2e2-47ee-a3ff-59f3e95edc1b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.254626 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.255732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.256257 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.256753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.257879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:06.258405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.260033 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.270514 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 464/907] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:06.271784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.276511 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.344292 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 444/908] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:05 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:06.345562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.378052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9193b79-a896-4b72-aa71-05d059f36f0b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.378900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.414704 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.415384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.415704 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.416064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.416812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:06.417214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.418641 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3e13c-1dd9-4aa4-b58b-d3a63d7e74e6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.428655 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 465/909] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:06.430091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbafe98-4e03-490a-9a24-1837c23d056a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.430354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbafe98-4e03-490a-9a24-1837c23d056a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.430639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbafe98-4e03-490a-9a24-1837c23d056a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.432174 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 466/910] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:06.433037 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbafe98-4e03-490a-9a24-1837c23d056a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.453022 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57dc17-aeda-4835-9581-e3e92045001b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.490459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.491290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.491572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.491844 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.502570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.504028 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.513685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-63ad7340-c4bc-442f-8dca-0fd1750cfa92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:06.524132 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:06.525261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:06.526926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:06.526926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:06.526926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:06.526926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:06.526926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 445/911] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:06.527290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.527290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.527591 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.546614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:06.547855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:06.564996 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbafe98-4e03-490a-9a24-1837c23d056a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.601460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.602128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.602413 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.602720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.603427 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.604812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.621548 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 446/912] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:06.622806 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.633424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 467/913] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:03:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:06.645623 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.651216 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.750647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf99695-25b9-48d8-b17e-f132daff0162 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.777009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab3ca-d6fd-4cc0-8406-f4b44c306776 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.798032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.800015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.800942 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.801600 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.802470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.803516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:06.804221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.806647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.818974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:06.819818 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:06.820099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:06.820370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:06.821041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:06.822525 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:06.830769 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 447/914] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:06.832100 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:06.841749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:06.842946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 15:03:06.843995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:06.848560 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 468/915] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:06.850463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:06.939340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:06.978038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65778e11-8a84-4f51-b3a3-7581177fff68 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:06.980036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86debdc-1a19-4bff-8a8b-de7e0682765d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:07.015623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.016435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.017084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.017398 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.024601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.026091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.031871 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.033092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.033605 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.034119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.035028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:07.035540 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.037110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:07.040816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:07.042126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:07.043243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 448/916] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:07.048844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 469/917] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:07.049509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:07.053054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:07.119910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:07.163468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630669-60ab-4e54-92c3-161cb8bd0e9a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:07.179828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb06f6-5f1e-4fc8-a367-8554c1d8eb04 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:07.203411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.204330 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.204763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.205071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.205918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:07.206207 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.207670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c588f2-b197-4d21-91c8-d1129a7ab99f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.209414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:07.219209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.219898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.220187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.220471 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.221204 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:07.221551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.222810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 449/918] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:07.222933 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047964fb-c900-4c62-9724-1c1aad16caff None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.224514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3dcb1-eb66-48f2-af30-996ea098b758 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.224808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3dcb1-eb66-48f2-af30-996ea098b758 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.225095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3dcb1-eb66-48f2-af30-996ea098b758 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.226519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 450/919] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:07.230394 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3dcb1-eb66-48f2-af30-996ea098b758 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:07.231760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 470/920] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:07.233265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.233737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.234184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.251029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:07.252254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:07.341260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 471/921] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:07.341260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:07.359924 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3dcb1-eb66-48f2-af30-996ea098b758 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:07.391359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:07.397392 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.398058 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.398346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.398622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.399468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] RBAC: Adding query filter params (role.id=a8673d7ff0a842fa9ec5f2dc9a293a02, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=daf0d21e592f4777becaaf853c52e043) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:07.399784 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.401274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.413552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 451/922] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:07 2022] GET /identity/v3/role_assignments?user.id=daf0d21e592f4777becaaf853c52e043&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=a8673d7ff0a842fa9ec5f2dc9a293a02 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:07.413552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:07.472194 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:07.475268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215ba4-b127-4716-8708-279a09ccb4f9 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:07.520980 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4ace9-fe02-4bd4-9f9f-59088ab9ac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.522180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ace9-fe02-4bd4-9f9f-59088ab9ac77 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.522699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ace9-fe02-4bd4-9f9f-59088ab9ac77 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.523085 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2733f-9cd2-47a9-9f2f-914c77e03662 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:07.523448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ace9-fe02-4bd4-9f9f-59088ab9ac77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.524449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4ace9-fe02-4bd4-9f9f-59088ab9ac77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.526418 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4ace9-fe02-4bd4-9f9f-59088ab9ac77 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.551374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 472/923] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:03:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:07.575823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86c648-e5f7-45d8-be67-de504d81821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:07.576623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86c648-e5f7-45d8-be67-de504d81821a None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:07.576911 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86c648-e5f7-45d8-be67-de504d81821a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:07.577268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86c648-e5f7-45d8-be67-de504d81821a None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:07.578093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86c648-e5f7-45d8-be67-de504d81821a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:07.579712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86c648-e5f7-45d8-be67-de504d81821a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:07.598612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 452/924] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:03:07 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:09.022433 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acde38e-b8ff-4c3e-822a-a6db6c133bdb None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.023015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acde38e-b8ff-4c3e-822a-a6db6c133bdb None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.023015 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acde38e-b8ff-4c3e-822a-a6db6c133bdb None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.024639 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 473/925] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:09.030008 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.030442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.030623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.048621 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:09.049826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:09.124134 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 453/926] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:09.148311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acde38e-b8ff-4c3e-822a-a6db6c133bdb None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:09.246192 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:09.250950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acde38e-b8ff-4c3e-822a-a6db6c133bdb None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:09.297012 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:09.298095 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.298618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.299107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.300045 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:09.301781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:09.303416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: WARNING keystone.server.flask.common [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 15:03:09.324738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.sql.core [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 15:03:09.324738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 15:03:09.324738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [parameters: {'id': '0bfbee5af3574ee3abb971127fca2676', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 15:03:09.324738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bfbee5af3574ee3abb971127fca2676.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bfbee5af3574ee3abb971127fca2676. May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application result = self._query(query) May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application conn.query(q) May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 15:03:09.342643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application result.read() May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 15:03:09.343726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application next(self.gen) May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application yield resource May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application next(self.gen) May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self.session.rollback() May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self.force_reraise() May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise self.value May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application session.commit() May 19 15:03:09.344974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._prepare_impl() May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self.session.flush() May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._flush(objects) May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application compat.raise_( May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application flush_context.execute() May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 15:03:09.346142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rec.execute(self) May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 15:03:09.347267 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application result = self._query(query) May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application conn.query(q) May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application result.read() May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 15:03:09.348245 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [parameters: {'id': '0bfbee5af3574ee3abb971127fca2676', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:09.350442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:09.352587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bfbee5af3574ee3abb971127fca2676. May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:09.356625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 474/927] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 19 15:03:09 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:09.366455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:09.406536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:09.477065 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6571-946e-4a4c-b636-46cc64e402d4 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:09.513155 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:09.514035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.514316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.514589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.522179 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:09.523766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d92cc464-fcb7-413e-a7d0-2248b3a2ffaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:09.537974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:09.539070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:09.540195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 454/928] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:09.545704 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:09.665485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d782f-a5df-4ec6-871c-8000abc8178e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:09.707633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:09.708354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.708640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.709001 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.709972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:09.710373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:09.712523 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c77520-df32-4caf-8813-d58946ab799e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:09.737194 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 475/929] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:09.811666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06d051-c745-4793-8b4a-8a222d864d0a None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.812440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06d051-c745-4793-8b4a-8a222d864d0a None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.812957 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06d051-c745-4793-8b4a-8a222d864d0a None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.815048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 455/930] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:09.822942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2198-95fa-4dde-855b-3f845b684e06 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.823643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2198-95fa-4dde-855b-3f845b684e06 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.824161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2198-95fa-4dde-855b-3f845b684e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.844209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-354c2198-95fa-4dde-855b-3f845b684e06 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:09.848033 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-354c2198-95fa-4dde-855b-3f845b684e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:09.861683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:09.895141 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ff60d-989e-457a-a942-482cc8527a94 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.895527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ff60d-989e-457a-a942-482cc8527a94 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.895712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ff60d-989e-457a-a942-482cc8527a94 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.897382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 456/931] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:09.903361 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.903716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.903927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.922062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:09.923292 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:09.928065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 476/932] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:09.930379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce38e-6ae8-43a8-8a0b-36326533f18d None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.930379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce38e-6ae8-43a8-8a0b-36326533f18d None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.930379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce38e-6ae8-43a8-8a0b-36326533f18d None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.931793 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 477/933] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:09.941598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:09.941889 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:09.942166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:09.961967 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:09.963193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:10.006148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 457/934] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:10.007012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.044483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 478/935] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:10.045935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.047358 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.129466 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.134941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac8f11-6232-485d-86c4-843893d8e697 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.158869 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0b90-eb38-4797-9f8f-7588d3432fd4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.174780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.175971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.176393 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.176687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.188941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:10.195823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.196649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.196930 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.197206 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] PATH_INFO: `/v3/services/compute` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:10.197490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:10.198588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:10.199610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:10.200403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.200487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 458/936] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:10.200487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.200630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:10.216397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:10.217974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:10.219159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 479/937] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:10.220726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.319246 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.333327 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a8731-26b3-441b-839f-402edd4fb1f3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.344727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e29ff-b0fc-47f0-a129-7fe10e4c1420 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.370096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.370902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.371187 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.371467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.372149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.373661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.384036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.384939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.385433 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.385899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.386794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:10.387261 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.388867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:10.389402 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:10.390789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:10.392729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:10.396279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 459/938] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:10.397140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.400001 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.408527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 480/939] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:10.409633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.510791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b25cf-f91b-4b24-b9d4-9643bbdd05f0 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.524361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ae525-fa24-48bc-b649-1ec632aaa530 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.547756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.548398 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.548692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.548997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.549752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:10.550038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.551579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.561294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 460/940] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:10.561294 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.568630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.569436 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.569788 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.570135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.570927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:10.571301 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.572784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.578664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 481/941] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:10.579402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.587161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.646812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.682396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c932-1e5e-4315-9cb3-6a8e02989205 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.687550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40a01f-c1dd-4c62-a2a4-49e8e38b7e81 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.720400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.721206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.721530 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.721797 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.726183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.727144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.727599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.728048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.729280 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] RBAC: Adding query filter params (type=compute) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:10.729690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.731482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.734071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.735861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.741653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 482/942] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:10.741653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:10.768097 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:10.769295 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:10.770404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:10.770404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:10.770404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:10.770404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:10.770404 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 461/943] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:10.775498 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.855085 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.857966 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bc8ed-7460-4583-8931-be8841d11d9c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.890192 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:10.899237 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53b30-0ffc-4f74-aa2f-e6aff5e1fdd1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:10.901087 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.902135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.902420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.902698 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.903442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:10.903755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.905329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.916334 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 483/944] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:10.918303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:10.944476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:10.945161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:10.945509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:10.945799 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:10.946555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:10.946873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:10.948299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:10.962807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 462/945] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:10.969880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:11.058540 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50d7dd-c76e-4b8d-b1c4-65049a955dad None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.108494 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832e19e-6036-4da7-856d-53f83bdbe0fe None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.114597 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.115607 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.116118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.117134 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.118210 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] RBAC: Adding query filter params (interface=public, service_id=45f716edfdbe4ba5b5977954cd074158, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:11.119269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.121490 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.134339 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 484/946] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/endpoints?service_id=45f716edfdbe4ba5b5977954cd074158&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:11.135072 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:11.145584 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:11.152637 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.153446 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.154203 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.154891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.163517 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.164988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:11.181408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:11.182580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:11.183723 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 463/947] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:03:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:11.189619 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:11.215074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:11.274399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7598e8-075d-4dcc-a077-bb9ea82d4877 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.305631 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dba20-cfd7-446b-9973-ffeac2bd8dce None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.325022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.325750 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.326098 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.326378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.327076 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.328497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.341725 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 485/948] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:03:11 2022] GET /identity/v3/services => generated 1497 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 15:03:11.352390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.353112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.353485 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.353775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.354678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:11.354996 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.356507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.380383 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 464/949] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 15:03:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:11.385789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:11.498991 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7af5d5-fed3-48b8-9e60-b1b48c24017e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.521659 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:11.536734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.538084 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.538620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.539300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.551409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.553064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:11.571173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:11.573071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:11.574207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:11.574207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:11.574207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:11.574207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:11.574207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 486/950] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:11.578579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:11.632453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:11.687806 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2b0db-8a2a-489d-ac5a-fadb1e7dd97a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.723914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.724585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.724871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.725151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.725925 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:11.726236 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.727692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.740191 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 465/951] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:11.745766 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:11.848037 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359bd6-d518-4b32-875a-b7c9f90c50d2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:11.882473 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:11.883296 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:11.883583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:11.883893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:11.891170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:11.893040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:11.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:11.909116 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:11.910408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 487/952] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:11.915384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:12.026121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c497db-9a14-4eda-a828-f055d13ce061 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:12.036840 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:12.062728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:12.063399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.063686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.063967 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.064777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:12.065089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:12.066565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47cc7f-aa11-470c-8c1c-da80c2b86e64 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:12.079728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 466/953] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:12.088540 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:12.134881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:12.149159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d02f6-c837-4028-8daf-0f7687d73b59 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.149766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d02f6-c837-4028-8daf-0f7687d73b59 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.149766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d02f6-c837-4028-8daf-0f7687d73b59 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.151538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 467/954] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:12.158239 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.158640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.159025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.176885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:12.178396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:12.199808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717ae62-5588-467f-901b-e3c8ac053bef None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:12.235441 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:12.236615 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.237058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.237543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/daf0d21e592f4777becaaf853c52e043/roles/a8673d7ff0a842fa9ec5f2dc9a293a02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.240295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=daf0d21e592f4777becaaf853c52e043, role_id=a8673d7ff0a842fa9ec5f2dc9a293a02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:12.242027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:12.261398 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 468/955] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:12.265172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 488/956] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:03:12 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/daf0d21e592f4777becaaf853c52e043/roles/a8673d7ff0a842fa9ec5f2dc9a293a02 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 15:03:12.284959 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:12.407440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:12.411667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456016-a6f2-4d07-abf1-5d264ae5549e None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:12.450339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:12.451084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.451371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.451658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.452383 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:12.453888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:12.455227 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: WARNING keystone.server.flask.common [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 15:03:12.468923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.sql.core [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 15:03:12.468923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 15:03:12.468923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [parameters: {'user_id': '3af0281da63c427e868776699bab4181', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 15:03:12.468923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277130) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result = self._query(query) May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application conn.query(q) May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result.read() May 19 15:03:12.476447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 15:03:12.477561 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application next(self.gen) May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application yield resource May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application next(self.gen) May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.session.rollback() May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.force_reraise() May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise self.value May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application session.commit() May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 15:03:12.479241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._prepare_impl() May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.session.flush() May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._flush(objects) May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application compat.raise_( May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application flush_context.execute() May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rec.execute(self) May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 15:03:12.480671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 15:03:12.481907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result = self._query(query) May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application conn.query(q) May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application result.read() May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 15:03:12.483245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [parameters: {'user_id': '3af0281da63c427e868776699bab4181', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:12.484505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 15:03:12.485766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:12.487172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 469/957] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 19 15:03:12 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:12.493878 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:12.600757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:12.610491 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce8dee-5c9f-45d5-97b3-353b482bcc1a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:12.648460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:12.649743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.650248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.650755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.659056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:12.660831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-12ed64b9-9786-4f22-b58e-3c50fae97548 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:12.676150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:12.677379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 15:03:12.678579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:12.685504 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 489/958] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:03:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:12.688063 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:12.797736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6037d99-b398-433f-a985-2b2e7726f1fb None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:12.833697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:12.834826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:12.835371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:12.835888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:12.836927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:12.837509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:12.839415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c17c-f872-4b4d-867f-304c99240b72 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:12.884598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 470/959] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 19 15:03:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:13.192625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce59ac-dc63-4b45-bf4b-b8dabcf19091 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:13.193210 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce59ac-dc63-4b45-bf4b-b8dabcf19091 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:13.193210 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce59ac-dc63-4b45-bf4b-b8dabcf19091 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:13.194805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 490/960] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:13.200115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49cf6d-5c8d-4578-a7eb-d75a4bd1e894 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:13.200525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49cf6d-5c8d-4578-a7eb-d75a4bd1e894 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:13.200699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49cf6d-5c8d-4578-a7eb-d75a4bd1e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:13.218099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-ca49cf6d-5c8d-4578-a7eb-d75a4bd1e894 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:13.219637 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-ca49cf6d-5c8d-4578-a7eb-d75a4bd1e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:13.234770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:13.300798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 471/961] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:13.323538 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce59ac-dc63-4b45-bf4b-b8dabcf19091 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:13.431913 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce59ac-dc63-4b45-bf4b-b8dabcf19091 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:13.468710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c2901-9e3e-46f5-96a9-b25ef8eec148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:13.469290 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2901-9e3e-46f5-96a9-b25ef8eec148 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:13.469618 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2901-9e3e-46f5-96a9-b25ef8eec148 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:13.469865 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2901-9e3e-46f5-96a9-b25ef8eec148 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:13.470628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2901-9e3e-46f5-96a9-b25ef8eec148 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:13.472013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2901-9e3e-46f5-96a9-b25ef8eec148 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:13.497272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 491/962] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:03:13 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:13.716551 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b76db-46e7-4578-a13c-dc32490574e4 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:13.716841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b76db-46e7-4578-a13c-dc32490574e4 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:13.717114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b76db-46e7-4578-a13c-dc32490574e4 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:13.718581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 472/963] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:13.724171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:13.724412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:13.724717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:13.741803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:13.743000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:13.751191 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:13.828394 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 492/964] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:13.845228 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b76db-46e7-4578-a13c-dc32490574e4 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:13.955192 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b76db-46e7-4578-a13c-dc32490574e4 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:13.989460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:13.990214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:13.990494 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:13.990766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:13.991423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:13.992806 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:14.005146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:14.006363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:14.007371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:14.009343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 473/965] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:14.012863 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:14.118518 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccfd8-219d-44f5-b5c8-386e7dd41127 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:14.156003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:14.156676 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:14.157086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:14.157221 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:14.157954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:14.158260 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:14.159643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:14.164843 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 493/966] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:14.170615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:14.186710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:14.286906 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f68ec-ebe9-4b94-b3c3-8d5f45269d01 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:14.322582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:14.323244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:14.323527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:14.323810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:14.324514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:14.324822 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:14.326346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066040ee-5e3e-4028-9b62-a037a0f60775 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:14.331577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 474/967] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:14.337375 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:14.340532 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:14.448130 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145d47-a09f-4780-b844-124acfb51f6b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:14.481929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b30b93-fda6-4218-960c-e60b5dce385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:14.482656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b30b93-fda6-4218-960c-e60b5dce385d None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:14.482936 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b30b93-fda6-4218-960c-e60b5dce385d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:14.483210 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b30b93-fda6-4218-960c-e60b5dce385d None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:14.483961 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b30b93-fda6-4218-960c-e60b5dce385d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:14.485428 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b30b93-fda6-4218-960c-e60b5dce385d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:14.505299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 494/968] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:03:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:14.577084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35901e91-bb8d-43f1-aed5-93ade2c291db None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:14.577324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35901e91-bb8d-43f1-aed5-93ade2c291db None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:14.577585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35901e91-bb8d-43f1-aed5-93ade2c291db None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:14.579181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 475/969] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:14.584931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:14.585186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:14.585527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:14.605424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:14.606648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:14.670804 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:14.694704 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 495/970] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:14.718761 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35901e91-bb8d-43f1-aed5-93ade2c291db None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:14.802309 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:14.828375 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35901e91-bb8d-43f1-aed5-93ade2c291db None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:14.863215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:14.864035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:14.864313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:14.864591 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:14.875559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:14.883554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:14.884653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:14.885736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 476/971] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 15:03:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:14.890742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:14.992021 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c47b5-f2a4-46cd-94fa-476e8454c327 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:15.026347 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:15.027029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.027304 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.027582 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.028332 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:15.028638 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:15.030135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:15.039806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 496/972] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:03:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:15.044399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:15.146212 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e64a97-b539-49ab-9d18-46969ac65a2f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:15.181047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:15.181895 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.182145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.182458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.185509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:15.193687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:15.195155 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:15.213874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:15.214977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:15.220945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:15.220945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:15.220945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:15.220945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:15.220945 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 477/973] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:15.221355 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:15.277555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:15.337897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfba75-2495-475a-aa53-e5a27273ac7e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:15.374896 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:15.375570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.375850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.376129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.376865 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:15.377175 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:15.378614 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:15.391891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 497/974] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:15.397667 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:15.498640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ad08e-f3d9-40cd-995a-97b63ac9c42f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:15.534535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:15.535349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.535636 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.535891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.543090 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:15.544574 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:15.559464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:15.560751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 15:03:15.562359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:15.565493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 478/975] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:03:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:15.566774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:15.673078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc8615-7ffd-405a-9c83-5997eb3c99cb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:15.681198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:15.709278 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:15.710017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.710298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.710571 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.711436 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:15.711767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:15.713305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f031d-0f6b-4754-850d-f87925388244 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:15.737342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 498/976] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 19 15:03:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:15.739211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:15.765487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3999ee-2798-4693-8fbb-c3cdd80bd741 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.765856 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3999ee-2798-4693-8fbb-c3cdd80bd741 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.766062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3999ee-2798-4693-8fbb-c3cdd80bd741 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.767842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 499/977] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:03:15.772847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929b985-5056-4685-836b-25a2499ade75 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.773257 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929b985-5056-4685-836b-25a2499ade75 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.773664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929b985-5056-4685-836b-25a2499ade75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.790758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-7929b985-5056-4685-836b-25a2499ade75 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:15.792099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-7929b985-5056-4685-836b-25a2499ade75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:15.798457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:15.855491 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7455d37-a2a6-4d36-8d24-18f979c7b418 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:15.872197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 500/978] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:15.893365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:15.894281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:15.894281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:15.894573 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:15.895246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929b985-5056-4685-836b-25a2499ade75 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:15.967951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:15.969363 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:15.995924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.003896 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929b985-5056-4685-836b-25a2499ade75 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.022586 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.042347 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.043419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.043693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.043978 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] PATH_INFO: `/v3/services/compute` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.044721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.046233 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:16.058306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:16.059724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:16.060927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 501/979] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:16.065710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.085401 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 479/980] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:15 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 15:03:16.090467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.178059 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e8775-c552-45f7-8e9e-6d6ba1995d6a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.192775 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305376f5-8beb-4b6c-bef5-66d48cfdc06e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.214971 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.215745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.216107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.216600 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.217438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:16.217927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.219442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.225172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 502/981] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:16.230690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.233181 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.234068 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.234352 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.234621 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.246587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.248121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.256743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.258738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:16.268000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:16.269114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:16.270384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:16.270384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:16.270384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:16.270384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:16.274374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 480/982] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:16.275241 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.352439 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a75d-5a79-4879-addb-181e19c763b7 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.385880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92025de2-f378-4fc3-aa67-2e73d90e2a78 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.402702 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.403773 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.404213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.404910 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.405749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] RBAC: Adding query filter params (type=compute) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:16.406142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.408123 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.414351 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 503/983] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:16.419806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.427344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.428104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.428462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.428809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.429660 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:16.430049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.431501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.448534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 481/984] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:16.449892 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.511282 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.511950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.529227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a50d7f-2544-46f1-8a52-0409e9de6e5b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.556657 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9551-7614-4561-806c-626c9fc1e350 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.565672 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.566341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.566638 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.566909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.567708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] RBAC: Adding query filter params (interface=public, service_id=794614e593924303980901ecc177c65a, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:16.568022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.569457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.580003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 504/985] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/endpoints?service_id=794614e593924303980901ecc177c65a&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:16.581630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.592150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.593045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.593558 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.593682 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.600661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.602152 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:16.617378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:16.618509 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:16.619633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 482/986] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:16.624370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.693231 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2100f-b366-4ce0-a4bb-23c7962bafcc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.730694 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.731379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.731669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.731977 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.732706 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.734162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.741756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644c9a7-c2e8-4833-b999-ec00e9ddcc2e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.749732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 505/987] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 15:03:16.767010 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.784841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.785691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.786037 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.786382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.787269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:16.787649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.789209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc158a95-622e-4d95-86fd-6be1ce66e7c9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:16.803345 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 483/988] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:16.809573 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.828872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:16.844747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ddab-2e24-4bc8-b370-5320b4abab00 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.845797 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ddab-2e24-4bc8-b370-5320b4abab00 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.846113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ddab-2e24-4bc8-b370-5320b4abab00 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.847817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 484/989] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:16.853674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.853983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.854249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.872542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:16.873935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:16.941515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e5f6-1340-45be-9d2f-60ddee862922 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:16.971699 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 485/990] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:16.992293 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:16.993674 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:16.993838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:16.993950 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:16.994253 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:16.995126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] RBAC: Adding query filter params (role.id=a8673d7ff0a842fa9ec5f2dc9a293a02, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=daf0d21e592f4777becaaf853c52e043) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:16.995441 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:16.996935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:17.003916 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 506/991] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/role_assignments?user.id=daf0d21e592f4777becaaf853c52e043&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=a8673d7ff0a842fa9ec5f2dc9a293a02 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:17.128047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575f48a-15d8-4f8d-8fb9-9661f4dcc699 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:17.142014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:17.172634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:17.173629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:17.173979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:17.174348 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:17.175092 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:17.176648 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:17.190012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:17.191287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:17.192380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:17.196732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 486/992] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:17.198348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:17.304496 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:17.327574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb4129-a6b1-428a-a146-2c5147914df1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:17.360835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:17.361672 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:17.361957 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:17.362237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:17.362939 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:17.363243 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:17.364643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:17.369560 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 507/993] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:17.375415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:17.477685 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c11e6-3efb-46d6-bcfb-a5fb5967005b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:17.511067 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:17.511741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:17.512015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:17.512288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:17.513000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:17.513445 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:17.514818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:17.520448 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 487/994] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:17.526447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:17.631795 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b71e46-8e21-4901-89f5-3b8ef163c05d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:17.666309 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:17.667112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:17.667265 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:17.667525 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:17.668317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] RBAC: Adding query filter params (interface=admin, service_id=45f716edfdbe4ba5b5977954cd074158, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:17.668628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:17.670076 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681cfab-af32-4d4d-893c-a62ea5ed511e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:17.675457 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 508/995] 10.222.0.73 () {56 vars in 1249 bytes} [Thu May 19 15:03:17 2022] GET /identity/v3/endpoints?service_id=45f716edfdbe4ba5b5977954cd074158&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:17.680948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:17.724174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:17.788716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a228dd-6c48-41ab-a941-318b2459f7a1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:17.824608 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a9b2d-10e7-4369-af25-6e32e0b9a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:17.825425 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a9b2d-10e7-4369-af25-6e32e0b9a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:17.825738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a9b2d-10e7-4369-af25-6e32e0b9a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:17.826055 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a9b2d-10e7-4369-af25-6e32e0b9a5d4 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:17.826754 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a9b2d-10e7-4369-af25-6e32e0b9a5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:17.828140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a9b2d-10e7-4369-af25-6e32e0b9a5d4 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:17.841426 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 488/996] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:03:17 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 15:03:19.019449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072d4aa-921c-44b8-bab0-c6634ed9b2a5 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.020047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072d4aa-921c-44b8-bab0-c6634ed9b2a5 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.020047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072d4aa-921c-44b8-bab0-c6634ed9b2a5 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.021597 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 509/997] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:03:19.026808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.027253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.027342 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.044823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:19.046044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:19.117738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 489/998] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:19.140741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072d4aa-921c-44b8-bab0-c6634ed9b2a5 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.151303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:19.256011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072d4aa-921c-44b8-bab0-c6634ed9b2a5 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.299549 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.300387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.300714 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.300996 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.301693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.303103 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-1a387162-7e46-4a34-b05d-b4986dab98a3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:19.315239 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:19.321501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 510/999] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:19.325293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.345870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253104b-71b2-41f0-bc22-8c05a5c6b842 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.346252 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253104b-71b2-41f0-bc22-8c05a5c6b842 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.346498 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253104b-71b2-41f0-bc22-8c05a5c6b842 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.348083 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 511/1000] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:19.353105 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.353394 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.353711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.359321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:19.371284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:19.372473 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:19.435427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0af57-d4e2-4ec1-9579-9702808b4f5a None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.435802 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:19.441352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 512/1001] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:19.463169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.472900 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.473713 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.473992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.474264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.475085 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:19.475392 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.476772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.482522 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 490/1002] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:19.488129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.568811 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba6719-13f2-416f-845a-0cb842a02dcd None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.589982 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:19.598555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe3a00-d9ca-4658-8ae6-87c3f73736f9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.604887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.605716 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.605997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.606269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] PATH_INFO: `/v3/services/image` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.607042 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.608930 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:19.622665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:19.624232 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:19.625470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 513/1003] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 15:03:19 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:19.629968 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.642377 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.643603 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.644050 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.644484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.645396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] RBAC: Adding query filter params (type=compute) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:19.645719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.647107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.656845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 491/1004] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 19 15:03:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:19.657996 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.702036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:19.742742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321ebaa-f552-4aa6-9774-537399bcf3a3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.762380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0811-ab3a-4989-ad17-269cc5825fdf None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.780544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.781369 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.781726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.782080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.782811 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] RBAC: Adding query filter params (name=image) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:19.783131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.784540 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.794651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 514/1005] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:19.796268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:19.804164 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ec3d-1d62-4dd2-b718-b53d6d835bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.804834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ec3d-1d62-4dd2-b718-b53d6d835bde None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.805565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ec3d-1d62-4dd2-b718-b53d6d835bde None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.805853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ec3d-1d62-4dd2-b718-b53d6d835bde None admin] PATH_INFO: `/v3/endpoints` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.806607 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72ec3d-1d62-4dd2-b718-b53d6d835bde None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.808691 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72ec3d-1d62-4dd2-b718-b53d6d835bde None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.829187 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 492/1006] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:03:19 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:19.923522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672116d-d5d9-45bf-bbbb-c08ada34d905 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:19.953846 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:19.970499 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:19.971370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:19.971746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:19.972112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:19.972932 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] RBAC: Adding query filter params (type=image) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:19.973407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:19.974946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87681cb5-407d-4606-88b4-0d4b8f6bb71d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:19.985742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 515/1007] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:20.066926 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62c651-b96c-4dcb-8be0-d75c565e0ce8 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:20.067496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62c651-b96c-4dcb-8be0-d75c565e0ce8 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:20.067496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62c651-b96c-4dcb-8be0-d75c565e0ce8 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:20.069116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 493/1008] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:20.075450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:20.076043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:20.076043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:20.095568 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:20.096764 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:20.173979 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 516/1009] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:20 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:20.197904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62c651-b96c-4dcb-8be0-d75c565e0ce8 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:20.217343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:20.318137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62c651-b96c-4dcb-8be0-d75c565e0ce8 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:20.356516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:20.357492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:20.357782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:20.358017 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:20.358726 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:20.360109 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:20.373121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:20.374492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 15:03:20.375577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:20.377302 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 494/1010] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 19 15:03:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:20.380827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:20.484295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4d3e3-f982-427a-b5e8-5488f17ae7de None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:20.503858 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:20.524517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:20.525654 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:20.525926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:20.526227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:20.526951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:20.527266 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:20.528668 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:20.534246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 517/1011] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:20.539884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:20.626661 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:20.649490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83236706-3766-4eec-ba88-ee21ddc10e7d None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:20.684696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:20.685506 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:20.685796 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:20.686077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:20.686859 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:20.687170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:20.688556 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3b999-24d5-4763-9775-eaf44548f9e8 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:20.694339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 495/1012] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 19 15:03:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:20.700006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:20.806925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc24bc-a923-4aa7-ba8f-43ef00bc0c23 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:20.843463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659c18e-fd2f-48d0-8320-c77e710a2d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:20.844198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c18e-fd2f-48d0-8320-c77e710a2d8f None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:20.844489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c18e-fd2f-48d0-8320-c77e710a2d8f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:20.844768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c18e-fd2f-48d0-8320-c77e710a2d8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:20.845575 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659c18e-fd2f-48d0-8320-c77e710a2d8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:20.847109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659c18e-fd2f-48d0-8320-c77e710a2d8f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:20.864902 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 518/1013] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:03:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:22.198463 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03f7b1-d7dd-44ec-ab09-c229e0e44b54 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.199099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03f7b1-d7dd-44ec-ab09-c229e0e44b54 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.199099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03f7b1-d7dd-44ec-ab09-c229e0e44b54 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.200547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 496/1014] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:22.204038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31989942-2c71-40e3-a542-24e2b2834c3b None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.204481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31989942-2c71-40e3-a542-24e2b2834c3b None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.204679 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31989942-2c71-40e3-a542-24e2b2834c3b None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.206252 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.206405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 519/1015] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:22.206559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.206559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.212992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.213303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.213588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.225467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:22.226671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:22.231027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:22.232387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:22.278695 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:22.307844 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 497/1016] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:22.313277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 520/1017] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:22.331229 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:22.334838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:22.404652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:22.439359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491da53a-c211-4ab7-b9da-221a6013d541 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:22.446959 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7ad48-40fe-4598-a8fa-1f45dc119f08 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:22.478841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:22.480009 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.480444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.480997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.484744 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:22.485464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.485751 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.486029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.486768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:22.488188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:22.493379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:22.501658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:22.503006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:22.504113 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 498/1018] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 15:03:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:22.506565 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 521/1019] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:03:22 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 15:03:22.515304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:22.634275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96a75-b55d-4a65-870f-d73baa3e83a4 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:22.667594 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:22.682516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:22.683209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.683496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.683772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.684540 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:22.684845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:22.686314 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:22.696587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 499/1020] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:03:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:22.702303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:22.738255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:22.819062 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385d8cc-c8dd-4599-81fb-9fc92d031123 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:22.854641 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:22.855506 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:22.855793 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:22.856076 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:22.866809 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:22.868337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:22.885632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:22.886707 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:22.888251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:22.888251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:22.888251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:22.888251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:22.892065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 522/1021] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:22.893780 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.005261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6b79b-e5e6-474b-8af3-ed2c474d76bc None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.040826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.041598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.041881 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.042156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.042878 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:23.043184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.044582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233306d6-6352-498d-b3e9-2b782fcfac57 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.056487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 500/1022] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:23.062217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.116268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabecd2a-64d3-47fd-b4a5-07c5fb6e4852 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.116609 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabecd2a-64d3-47fd-b4a5-07c5fb6e4852 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.116825 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabecd2a-64d3-47fd-b4a5-07c5fb6e4852 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.118307 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 501/1023] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:23.123682 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.123975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.124286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.142690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:23.143886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:23.154388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:23.173418 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ce2fe-3add-4037-9524-f423288e7f12 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.191467 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:23.209002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.209840 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.210124 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.210395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.218206 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.221282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 502/1024] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:23.221741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:23.238049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:23.239200 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:03:23.240357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:23.242633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.243942 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 523/1025] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:03:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:23.244959 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.355507 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2ae45-2600-4fc1-bafd-8f1b2f52d028 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.359666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dafa9-8c99-4986-93d0-d2c9ce4b64d9 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.393389 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.394085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.394376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.394660 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.395629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:23.396044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.396986 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.397555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.397832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.398116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.398389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.399595 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.401025 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.402939 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 503/1026] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:23.408622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.421475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 524/1027] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:03:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:23.422495 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.426165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:23.458833 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:23.524112 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1a904-2515-4e3b-976b-1d320a5cfd8c None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.534370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa49eb1-8700-4548-9fe6-9649e3af64c3 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.561255 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.561919 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.562199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.562475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.563301 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.564860 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.571486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.572161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.572450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.572737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.592159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 504/1028] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:03:23 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:23.661086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.662703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.665047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:23.776456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 525/1029] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:23.782370 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:23.888515 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:23.893756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f267776-f31d-4df1-bcf8-b60c9ab70c17 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:23.930038 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:23.931209 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:23.931705 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:23.932168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:23.943679 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:23.945383 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:23.963082 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:23.964206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:23.965318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:23.965318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:23.965318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:23.965318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:23.965318 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 505/1030] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:23.970872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:24.071690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e12e6f-2ded-4df0-b89a-f5a864647ac2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:24.073257 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:24.108041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:24.108715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:24.108992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:24.109300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:24.110019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:24.110325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:24.111725 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:24.123519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 526/1031] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:24.129131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:24.238955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5ff09-8e19-4caf-b07b-fd18bc6bac72 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:24.279138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:24.279960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:24.280244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:24.280517 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:24.288051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:24.289555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:24.304374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:24.305523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:24.306725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:24.309341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 506/1032] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:24.311959 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:24.418832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37037a18-5185-433b-9ead-ab10c5af993c None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:24.454488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:24.455446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:24.455898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:24.456333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:24.457403 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:24.457867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:24.459481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd9baf-eb71-4c3b-b04e-a911e628d104 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:24.472465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 527/1033] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:24.478544 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:24.494858 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:24.592822 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a32a0d-82d5-4fc1-a229-fc25672fec82 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:24.631826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:24.632500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:24.632784 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:24.633083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:24.633936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] RBAC: Adding query filter params (role.id=a8673d7ff0a842fa9ec5f2dc9a293a02, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=b764ac4c637749019a830d8cf3a22e09) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:24.634254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:24.635720 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4e9be-06ee-4aed-a900-687af439a5fc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:24.642012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 507/1034] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:24 2022] GET /identity/v3/role_assignments?user.id=b764ac4c637749019a830d8cf3a22e09&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=a8673d7ff0a842fa9ec5f2dc9a293a02 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:24.785899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a95ca0-840b-4274-9a3f-73fb616e3bf3 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:24.786125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a95ca0-840b-4274-9a3f-73fb616e3bf3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:24.786395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a95ca0-840b-4274-9a3f-73fb616e3bf3 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:24.788151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 528/1035] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:03:24.794088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:24.794413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:24.794740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:24.812999 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:24.814273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:24.893371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 508/1036] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:24.910175 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a95ca0-840b-4274-9a3f-73fb616e3bf3 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:24.936106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:25.019435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a95ca0-840b-4274-9a3f-73fb616e3bf3 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:25.055336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:25.056050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.056336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.056611 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] PATH_INFO: `/v3/services/image` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.057311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:25.058717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:25.070544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:25.071791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:25.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:25.076927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 529/1037] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 15:03:24 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:25.078023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:25.081447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:25.184853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66759f63-8c20-4b2c-9e43-6ce190908374 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:25.222140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:25.223217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.223749 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.224244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.225197 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] RBAC: Adding query filter params (name=image) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:25.225789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:25.227412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:25.232975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 509/1038] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:25.239084 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:25.339786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:25.355622 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593e293-f195-47be-9434-25dd36a26ad5 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:25.391339 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:25.392387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.392669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.392949 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.393698 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] RBAC: Adding query filter params (type=image) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:25.394007 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:25.395431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:25.400892 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 530/1039] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:25.406742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:25.475585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:25.512065 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73aa1-7c65-44f3-a69a-a270d45a5c37 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:25.548326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:25.548983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.549271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.549562 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.550359 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] RBAC: Adding query filter params (interface=public, service_id=9e18b045ff5b4441a29d9b883d984981, region_id=RegionOne) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:25.550670 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:25.552042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d2d0f-cdd4-4750-9344-850d65aa0c40 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:25.557504 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 510/1040] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:03:25 2022] GET /identity/v3/endpoints?service_id=9e18b045ff5b4441a29d9b883d984981&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:25.563113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:25.667720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39549-f4e8-40ef-b081-88b726912fb5 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:25.704142 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c07d6f-69ec-4918-a335-cf99e20bf3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:25.704848 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07d6f-69ec-4918-a335-cf99e20bf3aa None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.705137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07d6f-69ec-4918-a335-cf99e20bf3aa None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.705447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07d6f-69ec-4918-a335-cf99e20bf3aa None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.706167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c07d6f-69ec-4918-a335-cf99e20bf3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:25.707631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c07d6f-69ec-4918-a335-cf99e20bf3aa None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:25.721278 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 531/1041] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:03:25 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 15:03:25.904996 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452171ba-e8bc-4a5b-88da-91eebaf53dae None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.905428 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452171ba-e8bc-4a5b-88da-91eebaf53dae None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.905611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452171ba-e8bc-4a5b-88da-91eebaf53dae None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.906980 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 511/1042] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:25.912778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:25.913049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:25.913356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:25.932706 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:25.933883 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:25.944288 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:26.005640 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 532/1043] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:26.031091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452171ba-e8bc-4a5b-88da-91eebaf53dae None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:26.132041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452171ba-e8bc-4a5b-88da-91eebaf53dae None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:26.163029 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:26.169426 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2348c57-31f6-49c8-ab63-93326fcc25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:26.170235 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348c57-31f6-49c8-ab63-93326fcc25b9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:26.170520 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348c57-31f6-49c8-ab63-93326fcc25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:26.170795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348c57-31f6-49c8-ab63-93326fcc25b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:26.172018 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348c57-31f6-49c8-ab63-93326fcc25b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:26.173493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348c57-31f6-49c8-ab63-93326fcc25b9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:26.179768 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 512/1044] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:26.180947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:26.294736 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c21777-a03c-4f0c-8c73-336244af2d3b None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:26.330009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e717-4f3c-4af5-9336-639ac248f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:26.330691 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e717-4f3c-4af5-9336-639ac248f591 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:26.330975 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e717-4f3c-4af5-9336-639ac248f591 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:26.331248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e717-4f3c-4af5-9336-639ac248f591 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:26.331966 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e717-4f3c-4af5-9336-639ac248f591 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:26.333474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e717-4f3c-4af5-9336-639ac248f591 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:26.352002 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:26.365264 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 533/1045] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:03:26 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:26.865628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0028d53-89a4-4d14-9a2e-e00ddff9aea8 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:26.866259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0028d53-89a4-4d14-9a2e-e00ddff9aea8 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:26.866259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0028d53-89a4-4d14-9a2e-e00ddff9aea8 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:26.867885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 513/1046] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:26.873931 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:26.874203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:26.874545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:26.892489 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:26.893730 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:26.964087 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 534/1047] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:26.986634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0028d53-89a4-4d14-9a2e-e00ddff9aea8 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:27.092642 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0028d53-89a4-4d14-9a2e-e00ddff9aea8 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:27.127793 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:27.128532 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.128820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.129099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.139886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:27.148424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:27.149758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:27.150801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:27.153580 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 514/1048] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 15:03:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:27.155754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:27.259746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e98a5a-bd72-4063-a46a-b739b0d12ac3 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:27.293790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:27.301078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:27.302285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.302580 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.302860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.303629 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:27.303935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:27.305379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:27.319009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 535/1049] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:03:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:27.320183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:27.384478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:27.428440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17854af-3e78-455c-a022-5e11fcd75ddc None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:27.462901 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:27.463712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.463995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.464278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.475362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:27.476832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:27.494422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:27.495538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:27.501651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:27.501651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:27.501651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:27.501651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:27.501651 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 515/1050] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:27.501991 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:27.616823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976bfa-813c-4910-969d-a6f11497cb54 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:27.653580 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:27.654185 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.654478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.654732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.655483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:27.655796 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:27.657195 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:27.669984 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 536/1051] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:27.675513 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:27.775827 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d60a4-777d-4622-a984-b64d5d64045e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:27.790270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:27.817756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:27.818593 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.818859 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.819176 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.828226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:27.829742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-4657c577-207f-409c-8499-5e13828b5767 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:27.844039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:27.845178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:03:27.846800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:27.849538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 516/1052] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:03:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:27.851954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:27.906482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:27.955109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c34d-39c6-42f0-a9e8-67630eb64a6e None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:27.982585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb638a-bb16-44b6-829d-468a8ada59ce None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.983163 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb638a-bb16-44b6-829d-468a8ada59ce None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.983163 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb638a-bb16-44b6-829d-468a8ada59ce None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.984747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 517/1053] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 15:03:27.990054 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.990334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.990625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.992156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:27.992845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:27.993130 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:27.993444 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:27.994331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:27.994611 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:27.996074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.008296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:28.009740 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:28.014222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 537/1054] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:03:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:28.020182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.085287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 518/1055] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:28.105122 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.126574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ce539-4fe3-4065-913d-1e1c6ded2b42 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.138113 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.164082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.164810 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.165092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.165399 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.176535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.178152 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:28.196328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:28.197482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:28.198631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:28.198631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:28.198631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:28.198631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:28.198631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 538/1056] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:28.203134 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.213341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3507b-f5a8-4b5b-bfb6-c3ba5f9dade7 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.215052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.249441 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.250249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.250533 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.250810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] PATH_INFO: `/v3/services/image` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.251474 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.252868 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:28.265710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:28.267353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 15:03:28.268379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:28.272507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 519/1057] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:28.273690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.318759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60301-d345-4d43-9f4c-e0e5f8bc5c9d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.353447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.354109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.354393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.354643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.355401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:28.355713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.357272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.369054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 539/1058] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:28.375317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.379630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f604db-9f3e-413c-a684-5da40a12a027 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.400450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.417212 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.417869 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.418142 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.418451 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.419316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] RBAC: Adding query filter params (name=image) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:28.419649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.421146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.431397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 520/1059] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:28.432135 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.468447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.482129 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478a137-1463-42cd-8674-4dd335189e52 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.518789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.519595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.519893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.520173 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.527494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.529036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:28.544341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:28.545460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:28.547086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 540/1060] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:28.551422 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.553704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29642a9-e3be-4dca-8c83-b8e7775e3a94 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.592108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.593085 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.593969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.594415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.595310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] RBAC: Adding query filter params (type=image) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:28.596116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.598277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.604804 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 521/1061] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:28.611254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.644307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.665356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69236eb7-8b30-427c-8db5-46abfb32cac6 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.699897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.700557 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.700848 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.701131 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.701964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:28.702311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.703792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d31de-e08d-4e3b-bb15-6a9ce4fba1c5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.707969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.715835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55936b29-b3e7-4286-acde-3ec873e19778 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.716006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 541/1062] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:28.717284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012e317-ea25-459c-9c01-20288286acd7 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.717619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012e317-ea25-459c-9c01-20288286acd7 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.717890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012e317-ea25-459c-9c01-20288286acd7 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.719119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 542/1063] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:28.722054 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012e317-ea25-459c-9c01-20288286acd7 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.752293 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2150dd-c99d-41c2-a534-8f4bc93dda4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.752957 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2150dd-c99d-41c2-a534-8f4bc93dda4e None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.753272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2150dd-c99d-41c2-a534-8f4bc93dda4e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.753549 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2150dd-c99d-41c2-a534-8f4bc93dda4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.754324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2150dd-c99d-41c2-a534-8f4bc93dda4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.755738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2150dd-c99d-41c2-a534-8f4bc93dda4e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.775451 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 522/1064] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:03:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 15:03:28.776809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.777195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.777588 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.795296 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:28.796535 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:28.852605 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012e317-ea25-459c-9c01-20288286acd7 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:28.869344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 523/1065] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:28.892285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:28.893577 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:28.893738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:28.894182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:28.894617 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/b764ac4c637749019a830d8cf3a22e09/roles/a8673d7ff0a842fa9ec5f2dc9a293a02` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:28.897442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=b764ac4c637749019a830d8cf3a22e09, role_id=a8673d7ff0a842fa9ec5f2dc9a293a02)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:28.898952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:28.909767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:28.923093 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 543/1066] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:03:28 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/b764ac4c637749019a830d8cf3a22e09/roles/a8673d7ff0a842fa9ec5f2dc9a293a02 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:03:28.993880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:29.018674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc567354-0256-4b8d-9ea9-32b28dcda171 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:29.056077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:29.057646 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:29.058167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:29.058655 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:29.071375 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:29.079741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:29.080977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:29.082154 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 524/1067] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:29.090275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:29.202482 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795649e-9417-43a5-8fe4-b4abdaa708d1 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:29.237478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:29.238554 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:29.239065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:29.239597 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:29.240569 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:29.241100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:29.242729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:29.251549 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 544/1068] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:29.257531 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:29.373231 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7345448-dfec-48a6-b0b5-50ef0b5b3626 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:29.409725 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:29.410599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:29.410845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:29.411151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] PATH_INFO: `/v3/users/cf1c3fc16db24a61b199c76a97964fa5` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:29.422772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:29.426426 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] RBAC: Authorizing `identity:get_user(user_id=cf1c3fc16db24a61b199c76a97964fa5)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:29.427894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:29.430512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 525/1069] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:29 2022] GET /identity/v3/users/cf1c3fc16db24a61b199c76a97964fa5 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:29.435887 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:29.497517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:29.540026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf76dc-9497-4361-a285-efbbb5a9566a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:29.574530 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:29.575183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:29.575510 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:29.575753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:29.644954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:29.646449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:29.708180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:29.752085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 545/1070] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:29.757983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:29.861944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f03724-4d72-444f-a0f9-ac49bf4a5f5a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:29.908700 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb717f-647a-481d-a93a-57d3941ec92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:29.909611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb717f-647a-481d-a93a-57d3941ec92e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:29.909894 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb717f-647a-481d-a93a-57d3941ec92e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:29.910170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb717f-647a-481d-a93a-57d3941ec92e None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:29.911380 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb717f-647a-481d-a93a-57d3941ec92e None admin] RBAC: Authorizing `identity:get_project(project_id=368a6ba31ff54a4cb96cb337c151be29)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:29.912828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb717f-647a-481d-a93a-57d3941ec92e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:29.914768 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 526/1071] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:29 2022] GET /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:29.920921 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:30.026443 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4da55-4776-41b1-9c3f-3309fe1233e2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:30.063150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:30.063830 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:30.064117 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:30.064392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:30.065254 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=368a6ba31ff54a4cb96cb337c151be29, user.id=cf1c3fc16db24a61b199c76a97964fa5) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:30.065592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:30.067105 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce26ce8-5d2b-40cf-a9f1-ad17a6b03128 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:30.081331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 546/1072] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:29 2022] GET /identity/v3/role_assignments?user.id=cf1c3fc16db24a61b199c76a97964fa5&scope.project.id=368a6ba31ff54a4cb96cb337c151be29&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:31.064243 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b816-4bdb-402e-b367-0c4d53009eaf None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.064852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b816-4bdb-402e-b367-0c4d53009eaf None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.064852 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b816-4bdb-402e-b367-0c4d53009eaf None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.066400 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 527/1073] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:31.071885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.072329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.072431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.090488 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:31.091754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:31.110928 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.159974 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682c87b-53b9-47d8-b148-2b75b4bb0021 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.160311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682c87b-53b9-47d8-b148-2b75b4bb0021 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.160704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682c87b-53b9-47d8-b148-2b75b4bb0021 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.162301 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 528/1074] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:31.166007 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 547/1075] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:31.171704 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544b47-c8e3-430a-b533-352888922260 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.171950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544b47-c8e3-430a-b533-352888922260 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.172362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544b47-c8e3-430a-b533-352888922260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.188992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.190264 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-db544b47-c8e3-430a-b533-352888922260 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:31.191468 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-db544b47-c8e3-430a-b533-352888922260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:31.254979 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.271384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 529/1076] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:31.289728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db544b47-c8e3-430a-b533-352888922260 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.306198 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9d6eb-1074-41e0-a9a2-6d3481b89506 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.341283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.341875 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.342160 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.342437 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.366697 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.396413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db544b47-c8e3-430a-b533-352888922260 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.417644 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:31.419194 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:31.438084 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.438883 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.439169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.439614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.449915 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:31.458183 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:31.459410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:31.462647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 530/1077] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:31.465336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.525458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 548/1078] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:31.525694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.532588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.580516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844d9d-91ce-40cc-bf56-466f6df655d9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.593479 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.617149 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.617848 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.618148 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.618427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.619172 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:31.619482 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:31.620861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:31.633288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 531/1079] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:31.635304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.645207 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7e48e-612f-4d19-a914-a2cb352aafcd None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.684447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.685505 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.685914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.686197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.699974 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:31.701732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:31.722126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:31.723329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:31.729535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:31.729535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:31.729535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:31.729535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:31.729535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 549/1080] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:31.731792 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.743501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce178856-8446-4c5a-919e-1a0edfc23672 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.761341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.785916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.787310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.787774 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.788226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.803923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:31.805696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:31.823620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:31.824732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:31.825861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:31.825861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:31.825861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:31.825861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:31.825861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 532/1081] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:31.830503 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:31.845728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be5250-19f7-48d1-bdac-296ebeaa76a2 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.855274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:31.886056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.886823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.887112 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.887417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.888150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:31.888460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:31.890189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:31.913283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 550/1082] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:31.948864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b9823-c8ee-4d14-a0d0-d35c60eecd31 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:31.988286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:31.989055 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:31.989379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:31.989631 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:31.990390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:31.990701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:31.992127 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.007320 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 533/1083] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:32.013167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.126380 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e16b3-f16f-4a86-9d2c-f5501a00ae19 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.165035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.165905 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.166179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.166454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.174665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:32.176127 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-d5a296c0-3f0c-4a36-9ad4-4751ffeb0921 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:03:32.190345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:32.191693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:32.193373 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 551/1084] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:32.197479 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.233785 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.304238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccfba0-3520-4213-b3f2-ff4e2805c272 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.304528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccfba0-3520-4213-b3f2-ff4e2805c272 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.304812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccfba0-3520-4213-b3f2-ff4e2805c272 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.306273 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 552/1085] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:32.311819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.312111 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.314501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc58732-322c-41ba-9185-b48e88be6a28 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.314643 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.329376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:32.330594 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:32.350081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.350751 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.351031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.351288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.352139 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.352270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:32.352418 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:32.353924 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.372262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 534/1086] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:32.377817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.401197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 553/1087] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:03:32.423953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.467734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.480762 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf5ac-7913-4a6c-96ea-818dc1f254b9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.515735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3609222-6304-452c-971c-825d1de65818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.516388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3609222-6304-452c-971c-825d1de65818 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.517032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3609222-6304-452c-971c-825d1de65818 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.517343 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3609222-6304-452c-971c-825d1de65818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.523966 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe82e0-1d51-4e55-9236-09728d2d8c2b None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.560648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.561487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.561778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.562056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.572516 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:32.580263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:32.581543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:03:32.582566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 554/1088] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:32.584081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3609222-6304-452c-971c-825d1de65818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:32.585734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3609222-6304-452c-971c-825d1de65818 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.587558 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.605345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.672719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.686773 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 535/1089] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:32.688176 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dda4b7-8658-4158-93e4-35b2a574c1a0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.692628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3609222-6304-452c-971c-825d1de65818 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.722139 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.722838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.723119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.723395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.724145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:32.724452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:32.725876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.734042 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 555/1090] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:32.740000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.797223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3609222-6304-452c-971c-825d1de65818 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.819153 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.832460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.833372 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.833664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.833949 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.841587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb8f8d-1053-46fc-8722-4cff8d9accfc None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.844341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:32.845857 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:03:32.863282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:32.864462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:32.865622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:03:32.865622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:03:32.865622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:03:32.865622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:32.865622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 536/1091] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:32.870237 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.878064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:32.878954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:32.879241 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:32.879519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] PATH_INFO: `/v3/users/cf1c3fc16db24a61b199c76a97964fa5` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:32.880746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1c3fc16db24a61b199c76a97964fa5)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:32.882401 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:32.884065 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 556/1092] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/users/cf1c3fc16db24a61b199c76a97964fa5 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:32.889911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:32.912396 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:32.969389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc876661-ce81-4c96-813d-8f6cca1c308b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:32.992976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfba97b-bfeb-4e99-8af3-e5f7c8ed0795 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:33.002472 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:33.003136 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:33.003424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:33.003697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:33.004419 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:33.004735 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:33.006210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:33.019791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 537/1093] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:33.025325 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:33.030382 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:33.031262 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:33.031546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:33.031820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:33.033011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] RBAC: Authorizing `identity:get_project(project_id=368a6ba31ff54a4cb96cb337c151be29)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:33.034492 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:33.036192 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 557/1094] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:32 2022] GET /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:33.042229 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:33.060057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:33.125403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:33.128650 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9ed97-b96b-48f0-b6c6-722255a2f186 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:33.145998 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f330d-10f3-44a5-a57e-6d58c635ebbb None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:33.162250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:33.163051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:33.163340 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:33.163620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:33.170405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:33.171872 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:33.179917 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:33.180824 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:33.181106 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:33.181416 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29/users/cf1c3fc16db24a61b199c76a97964fa5/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:33.183390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] RBAC: Authorizing `identity:create_grant(project_id=368a6ba31ff54a4cb96cb337c151be29, user_id=cf1c3fc16db24a61b199c76a97964fa5, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:33.184864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:03:33.186220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:33.187508 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:33.188715 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 538/1095] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:03:33.193179 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:33.203018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 558/1096] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:03:33 2022] PUT /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29/users/cf1c3fc16db24a61b199c76a97964fa5/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 15:03:33.308091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a3233-66b1-4ee0-9485-c62d8d188697 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:33.345860 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:33.346546 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:33.346831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:33.347110 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:33.347934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:33.348247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:33.349760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376dc13-39a8-4c3c-a45c-4c0c82eb7b80 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:33.363262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 539/1097] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:03:33.369713 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:33.449550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:33.477149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814c1e1-ed14-4605-88ae-745686e58c01 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:33.513310 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:33.514271 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:33.514550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:33.514805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:33.515656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] RBAC: Adding query filter params (role.id=a8673d7ff0a842fa9ec5f2dc9a293a02, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=b764ac4c637749019a830d8cf3a22e09) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:33.515972 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:33.517831 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a46070-d46e-4e5f-8e7a-48281d1ad5c9 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:33.524093 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 559/1098] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:33 2022] GET /identity/v3/role_assignments?user.id=b764ac4c637749019a830d8cf3a22e09&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=a8673d7ff0a842fa9ec5f2dc9a293a02 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:35.430567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4ee91-f8b7-4663-85ec-9841dfa6f6f9 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:35.431480 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4ee91-f8b7-4663-85ec-9841dfa6f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:35.431480 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4ee91-f8b7-4663-85ec-9841dfa6f6f9 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:35.433792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 540/1099] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:35.439398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:35.439858 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:35.439969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:35.458385 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:35.459587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:35.528951 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 560/1100] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:35.553057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4ee91-f8b7-4663-85ec-9841dfa6f6f9 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:35.580674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:35.672620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4ee91-f8b7-4663-85ec-9841dfa6f6f9 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:35.708036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:35.708821 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:35.709106 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:35.709420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:35.721390 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:35.728934 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:35.730286 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:35.731696 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 541/1101] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:35.736494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:35.807190 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:35.842052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056767-7a5f-499c-9a1b-b96b40dc9e6a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:35.876603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:35.877283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:35.877588 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:35.877836 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:35.878625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:35.878938 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:35.880343 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:35.888826 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 561/1102] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:35.895326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:35.980274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:36.000806 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73bcf4-9e2f-474f-8f9f-7091fb41ed47 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:36.036738 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:36.037628 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:36.037913 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:36.038192 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] PATH_INFO: `/v3/users/cf1c3fc16db24a61b199c76a97964fa5` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:36.039452 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] RBAC: Authorizing `identity:get_user(user_id=cf1c3fc16db24a61b199c76a97964fa5)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:36.040929 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:36.042599 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 542/1103] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:35 2022] GET /identity/v3/users/cf1c3fc16db24a61b199c76a97964fa5 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:36.049119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:36.164314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2aaa9-525f-4974-9594-113f2ae6670f None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:36.192590 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:36.199911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:36.200614 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:36.200899 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:36.201179 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:36.273128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:36.274653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:36.379337 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 562/1104] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:36.386156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:36.505225 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8252326-ac3f-4953-836d-65089c705a8f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:36.545493 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e9b7e-9d0c-42b3-b4b6-2078fae93f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:36.546360 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9b7e-9d0c-42b3-b4b6-2078fae93f75 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:36.546648 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9b7e-9d0c-42b3-b4b6-2078fae93f75 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:36.546927 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9b7e-9d0c-42b3-b4b6-2078fae93f75 None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:36.548173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9b7e-9d0c-42b3-b4b6-2078fae93f75 None admin] RBAC: Authorizing `identity:get_project(project_id=368a6ba31ff54a4cb96cb337c151be29)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:36.549655 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9b7e-9d0c-42b3-b4b6-2078fae93f75 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:36.551387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 543/1105] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:36 2022] GET /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:36.557781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:36.592449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:36.675860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7c64-6b8e-4fb7-ae9d-ed99f3971cbd None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:36.712026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:36.713026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:36.713529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:36.713880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:36.715170 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=368a6ba31ff54a4cb96cb337c151be29, user.id=cf1c3fc16db24a61b199c76a97964fa5) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:36.715596 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:36.717978 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ad346-247f-4156-9e16-b321ac238964 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:36.727785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 563/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:36 2022] GET /identity/v3/role_assignments?user.id=cf1c3fc16db24a61b199c76a97964fa5&scope.project.id=368a6ba31ff54a4cb96cb337c151be29&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:39.001166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb60d6-d295-4a30-8d82-152c1b21d501 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:39.002174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb60d6-d295-4a30-8d82-152c1b21d501 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:39.002174 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb60d6-d295-4a30-8d82-152c1b21d501 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:39.004312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 544/1107] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:39.010889 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:39.011654 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:39.011847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:39.038587 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:39.040397 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:39.111285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 564/1108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:39.143980 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb60d6-d295-4a30-8d82-152c1b21d501 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:39.165045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:39.260281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb60d6-d295-4a30-8d82-152c1b21d501 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:39.299001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c4d11-2ad6-4f64-95a8-7afac1f1237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:39.299805 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c4d11-2ad6-4f64-95a8-7afac1f1237a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:39.300091 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c4d11-2ad6-4f64-95a8-7afac1f1237a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:39.300368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c4d11-2ad6-4f64-95a8-7afac1f1237a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:39.301736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c4d11-2ad6-4f64-95a8-7afac1f1237a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:39.303195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c4d11-2ad6-4f64-95a8-7afac1f1237a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:39.304859 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 545/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:39.311048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:39.338839 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:39.421414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4acf-e158-4ca9-9dcd-ef51dd2861e4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:39.457354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf474-11af-4ff9-8798-f46d396d23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:39.458023 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf474-11af-4ff9-8798-f46d396d23b5 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:39.458302 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf474-11af-4ff9-8798-f46d396d23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:39.458583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf474-11af-4ff9-8798-f46d396d23b5 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:39.459300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf474-11af-4ff9-8798-f46d396d23b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:39.460754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf474-11af-4ff9-8798-f46d396d23b5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:39.482555 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 565/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:03:39 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:41.732226 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68354873-b6ed-42bf-b611-e339b81b8856 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:41.733193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68354873-b6ed-42bf-b611-e339b81b8856 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:41.733193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68354873-b6ed-42bf-b611-e339b81b8856 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:41.735353 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 546/1111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:41.744156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:41.744583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:41.744743 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:41.763944 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:41.765161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:41.836881 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 566/1112] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:41.870020 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68354873-b6ed-42bf-b611-e339b81b8856 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:41.973418 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:41.992020 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68354873-b6ed-42bf-b611-e339b81b8856 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:42.036512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:42.037864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:42.038245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:42.038615 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] PATH_INFO: `/v3/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:42.041316 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] RBAC: Authorizing `identity:get_role(role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:42.043511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:42.045107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 547/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:41 2022] GET /identity/v3/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:42.051182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:42.063226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:42.155497 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58cb01-4fdd-4463-bab7-00ff21acb5ee None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:42.190657 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:42.191522 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:42.191807 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:42.192081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] PATH_INFO: `/v3/users/4ffb3d19bc8c4c81af8ce346daae3711` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:42.206779 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] RBAC: Authorizing `identity:get_user(user_id=4ffb3d19bc8c4c81af8ce346daae3711)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:42.208256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:42.210011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 567/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:42 2022] GET /identity/v3/users/4ffb3d19bc8c4c81af8ce346daae3711 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:42.216043 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:42.326500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e897ae-f83b-4ac5-b07b-3c43b236afe1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:42.368472 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:42.369453 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:42.369770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:42.370101 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:42.398990 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:42.451511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:42.452986 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:42.557397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 548/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:42.564384 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:42.650786 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:42.686748 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff667bb-c6d4-4cba-95e6-32a755524602 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:42.723861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594aa6-ff63-47e7-8642-8cf06d0acfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:42.724730 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594aa6-ff63-47e7-8642-8cf06d0acfe2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:42.725010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594aa6-ff63-47e7-8642-8cf06d0acfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:42.725277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594aa6-ff63-47e7-8642-8cf06d0acfe2 None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:42.726636 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91594aa6-ff63-47e7-8642-8cf06d0acfe2 None admin] RBAC: Authorizing `identity:get_project(project_id=368a6ba31ff54a4cb96cb337c151be29)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:42.728088 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91594aa6-ff63-47e7-8642-8cf06d0acfe2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:42.729832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 568/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:42 2022] GET /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:42.737114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:42.802399 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:42.860869 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336f865-b3dc-4851-ba66-df8029501fb2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:42.897491 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:42.898139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:42.898428 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:42.898703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:42.899548 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] RBAC: Adding query filter params (role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=368a6ba31ff54a4cb96cb337c151be29, user.id=4ffb3d19bc8c4c81af8ce346daae3711) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:42.899864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:42.901345 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bd62-c0bc-4274-98ea-21363d7a8ce1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:42.907107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 549/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:42 2022] GET /identity/v3/role_assignments?user.id=4ffb3d19bc8c4c81af8ce346daae3711&scope.project.id=368a6ba31ff54a4cb96cb337c151be29&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:45.090778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a58b1a-70db-4181-b730-95e857cd56d5 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:45.091711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a58b1a-70db-4181-b730-95e857cd56d5 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:45.091711 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a58b1a-70db-4181-b730-95e857cd56d5 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:45.094036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 569/1118] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:45.100365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:45.100820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:45.100922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:45.120818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:45.122140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:45.192993 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 550/1119] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:45.218513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a58b1a-70db-4181-b730-95e857cd56d5 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:45.332837 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a58b1a-70db-4181-b730-95e857cd56d5 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:45.368893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:45.369797 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:45.370080 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:45.370354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] PATH_INFO: `/v3/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:45.371885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] RBAC: Authorizing `identity:get_role(role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:45.373356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:45.374747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 570/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:45 2022] GET /identity/v3/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:45.380853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:45.429633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:45.488484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5517b82-c5ea-4bef-a385-6be12e4d4a24 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:45.531888 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:45.532782 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:45.533070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:45.533375 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] PATH_INFO: `/v3/users/4ffb3d19bc8c4c81af8ce346daae3711` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:45.534830 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] RBAC: Authorizing `identity:get_user(user_id=4ffb3d19bc8c4c81af8ce346daae3711)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:45.536308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:45.538039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 551/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:45 2022] GET /identity/v3/users/4ffb3d19bc8c4c81af8ce346daae3711 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:45.547006 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:45.551247 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:45.654255 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e780bee-9cd8-4780-aaed-5eb0c93e7914 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:45.689533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4779a2-ee10-4ea1-a5b2-d09a8323dd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:45.690388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4779a2-ee10-4ea1-a5b2-d09a8323dd68 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:45.690671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4779a2-ee10-4ea1-a5b2-d09a8323dd68 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:45.690940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4779a2-ee10-4ea1-a5b2-d09a8323dd68 None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:45.692256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4779a2-ee10-4ea1-a5b2-d09a8323dd68 None admin] RBAC: Authorizing `identity:get_project(project_id=368a6ba31ff54a4cb96cb337c151be29)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:45.693731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4779a2-ee10-4ea1-a5b2-d09a8323dd68 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:45.695407 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 571/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:45 2022] GET /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:45.701850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:45.818162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe675a9-c7e1-4c81-9ceb-d6708cee44f6 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:45.833070 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:45.863096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8641423-5e23-4492-b3a7-f46e142e8bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:45.863981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8641423-5e23-4492-b3a7-f46e142e8bff None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:45.864274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8641423-5e23-4492-b3a7-f46e142e8bff None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:45.864554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8641423-5e23-4492-b3a7-f46e142e8bff None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29/users/4ffb3d19bc8c4c81af8ce346daae3711/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:45.867004 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8641423-5e23-4492-b3a7-f46e142e8bff None admin] RBAC: Authorizing `identity:create_grant(project_id=368a6ba31ff54a4cb96cb337c151be29, user_id=4ffb3d19bc8c4c81af8ce346daae3711, role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:45.868461 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8641423-5e23-4492-b3a7-f46e142e8bff None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:45.883403 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 552/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:03:45 2022] PUT /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29/users/4ffb3d19bc8c4c81af8ce346daae3711/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:03:48.046340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dd0ae-b48d-4d31-97be-43befdf77dda None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:48.047275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dd0ae-b48d-4d31-97be-43befdf77dda None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:48.047275 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dd0ae-b48d-4d31-97be-43befdf77dda None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:48.049438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 572/1124] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:48.054895 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:48.055347 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:48.055459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:48.074378 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:48.075620 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:48.145936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 553/1125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:48.169882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dd0ae-b48d-4d31-97be-43befdf77dda None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:48.259233 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:48.297500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dd0ae-b48d-4d31-97be-43befdf77dda None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:48.332441 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:48.333363 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:48.333649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:48.333922 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] PATH_INFO: `/v3/roles/f043b9bbc0714db58a8ec27b71ee493f` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:48.335546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] RBAC: Authorizing `identity:get_role(role_id=f043b9bbc0714db58a8ec27b71ee493f)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:48.336947 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:48.338387 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 573/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:48 2022] GET /identity/v3/roles/f043b9bbc0714db58a8ec27b71ee493f => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:48.345604 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:48.459549 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459a94-4e21-4671-b211-9e5239798761 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:48.461885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:48.496435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:48.497431 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:48.497718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:48.498001 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] PATH_INFO: `/v3/users/4ffb3d19bc8c4c81af8ce346daae3711` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:48.499256 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] RBAC: Authorizing `identity:get_user(user_id=4ffb3d19bc8c4c81af8ce346daae3711)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:48.500717 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:48.502405 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 554/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:48 2022] GET /identity/v3/users/4ffb3d19bc8c4c81af8ce346daae3711 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:48.508419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:48.618838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c04d7a-a7c5-4701-854a-a9c6fac71640 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:48.655249 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:48.655909 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:48.656192 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:48.656466 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:48.666995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:48.731824 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:48.733327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:48.835456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 574/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:48.842368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:48.950770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4266-8136-4da0-80e6-30ebe0dea326 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:48.987435 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678524a-950d-4425-a7ae-7bc692dcace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:48.988301 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678524a-950d-4425-a7ae-7bc692dcace6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:48.988587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678524a-950d-4425-a7ae-7bc692dcace6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:48.988867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678524a-950d-4425-a7ae-7bc692dcace6 None admin] PATH_INFO: `/v3/projects/368a6ba31ff54a4cb96cb337c151be29` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:48.990156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678524a-950d-4425-a7ae-7bc692dcace6 None admin] RBAC: Authorizing `identity:get_project(project_id=368a6ba31ff54a4cb96cb337c151be29)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:48.991607 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678524a-950d-4425-a7ae-7bc692dcace6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:48.993384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 555/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:48 2022] GET /identity/v3/projects/368a6ba31ff54a4cb96cb337c151be29 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:49.003222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:49.050295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:49.111141 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456ab9c-2d8d-4a43-81ec-d48b99bf6e17 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:49.146323 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:49.146969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:49.147243 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:49.147512 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:49.148340 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] RBAC: Adding query filter params (role.id=f043b9bbc0714db58a8ec27b71ee493f, scope.project.id=368a6ba31ff54a4cb96cb337c151be29, user.id=4ffb3d19bc8c4c81af8ce346daae3711) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:49.148647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:49.150149 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4c0a-8fc3-4a1b-9fc1-f761549a6180 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:49.156208 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 575/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:49 2022] GET /identity/v3/role_assignments?user.id=4ffb3d19bc8c4c81af8ce346daae3711&scope.project.id=368a6ba31ff54a4cb96cb337c151be29&role.id=f043b9bbc0714db58a8ec27b71ee493f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:51.364239 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7aab1-8932-4f09-a00b-449f7bcabd80 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:51.365083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7aab1-8932-4f09-a00b-449f7bcabd80 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:51.365083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7aab1-8932-4f09-a00b-449f7bcabd80 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:51.367481 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 556/1131] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:51.372994 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:51.373493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:51.373606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:51.392800 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:51.394038 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:51.468619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 576/1132] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:51.491583 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7aab1-8932-4f09-a00b-449f7bcabd80 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:51.543758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:51.611381 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7aab1-8932-4f09-a00b-449f7bcabd80 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:51.647966 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b5e9e-d153-4082-bfda-8c2e5beab360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:51.648757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b5e9e-d153-4082-bfda-8c2e5beab360 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:51.649041 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b5e9e-d153-4082-bfda-8c2e5beab360 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:51.649345 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b5e9e-d153-4082-bfda-8c2e5beab360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:51.650669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b5e9e-d153-4082-bfda-8c2e5beab360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:51.652126 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b5e9e-d153-4082-bfda-8c2e5beab360 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:51.653842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 557/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:51.662770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:51.700199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:51.777408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7cb5d-020c-4030-b54c-98f2364dff98 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:51.816040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d387d-af27-4f7a-bf94-941d705a0d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:51.816680 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d387d-af27-4f7a-bf94-941d705a0d3b None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:51.816963 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d387d-af27-4f7a-bf94-941d705a0d3b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:51.817276 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d387d-af27-4f7a-bf94-941d705a0d3b None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:51.817983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d387d-af27-4f7a-bf94-941d705a0d3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:51.819441 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d387d-af27-4f7a-bf94-941d705a0d3b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:51.849529 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 577/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:03:51 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:03:54.090007 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fb37c-3231-4dc5-a979-0d98f9645780 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:54.090899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fb37c-3231-4dc5-a979-0d98f9645780 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:54.090899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fb37c-3231-4dc5-a979-0d98f9645780 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:54.093173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 558/1135] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:54.099565 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:54.100016 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:54.100135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:54.120812 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:54.122061 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:54.193320 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 578/1136] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:54.220954 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fb37c-3231-4dc5-a979-0d98f9645780 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:54.332698 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fb37c-3231-4dc5-a979-0d98f9645780 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:54.342168 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:54.373299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b646c-2afc-4982-a731-ccf4bd83fe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:54.374129 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b646c-2afc-4982-a731-ccf4bd83fe57 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:54.374418 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b646c-2afc-4982-a731-ccf4bd83fe57 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:54.374697 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b646c-2afc-4982-a731-ccf4bd83fe57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:54.375963 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b646c-2afc-4982-a731-ccf4bd83fe57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:54.377478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b646c-2afc-4982-a731-ccf4bd83fe57 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:54.379115 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 559/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:03:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:54.385746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:54.409728 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:54.495751 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85297f-7374-41c8-a5f3-6f0239070558 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:54.534099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b088076-a67e-4d40-a8b3-ec37b87b117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:54.534763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b088076-a67e-4d40-a8b3-ec37b87b117a None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:54.535047 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b088076-a67e-4d40-a8b3-ec37b87b117a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:54.535300 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b088076-a67e-4d40-a8b3-ec37b87b117a None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:54.536010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b088076-a67e-4d40-a8b3-ec37b87b117a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:54.537496 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b088076-a67e-4d40-a8b3-ec37b87b117a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:54.560518 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 579/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:03:54 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 15:03:56.794943 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bfe2d-b103-4aea-9c3b-c1021676a2c8 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:56.795536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bfe2d-b103-4aea-9c3b-c1021676a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:56.795536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bfe2d-b103-4aea-9c3b-c1021676a2c8 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:56.796891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 560/1139] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:03:56.803513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:56.803962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:56.804081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:56.825064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:03:56.826305 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:03:56.899656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 580/1140] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:03:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:03:56.931204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bfe2d-b103-4aea-9c3b-c1021676a2c8 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:57.049010 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bfe2d-b103-4aea-9c3b-c1021676a2c8 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:57.085659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:57.086442 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:57.086729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:57.087008 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:57.099411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:03:57.107064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:03:57.108379 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:03:57.109744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 561/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:03:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:03:57.115196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:57.118487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:57.233470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a02f2d-29b1-4787-a5b4-c51cc8c92a02 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:57.269299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:57.270020 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:57.270269 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:57.270574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:57.271327 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:57.271598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:57.273017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:57.281888 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 581/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:03:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:57.287994 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:57.326983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:57.402136 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bd0e6-6da2-4478-882a-6079e6cf48d8 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:57.437392 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:57.438256 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:57.438537 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:57.438809 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] PATH_INFO: `/v3/users/c52d323cd4064a21ac2845a2f4a09635` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:57.453936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] RBAC: Authorizing `identity:get_user(user_id=c52d323cd4064a21ac2845a2f4a09635)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:57.455386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:57.457037 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 562/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:03:57 2022] GET /identity/v3/users/c52d323cd4064a21ac2845a2f4a09635 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:57.466334 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:57.542270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:57.588981 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b57ec-dbe1-4633-8443-1807a95bca76 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:57.625683 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:57.626332 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:57.626615 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:57.626895 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:57.698458 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:57.699937 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:57.762829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:57.806801 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 582/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:03:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:03:57.813083 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:57.913961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:03:57.921565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5dc22-18ae-47f3-a08c-13cc4f26f687 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:57.960447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ef0c3-c0e0-43c4-b286-f803aea7dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:57.961339 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef0c3-c0e0-43c4-b286-f803aea7dd73 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:57.961630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef0c3-c0e0-43c4-b286-f803aea7dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:57.961914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef0c3-c0e0-43c4-b286-f803aea7dd73 None admin] PATH_INFO: `/v3/projects/9ae4af84f05445aca485ecae61133bd3` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:57.963258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ef0c3-c0e0-43c4-b286-f803aea7dd73 None admin] RBAC: Authorizing `identity:get_project(project_id=9ae4af84f05445aca485ecae61133bd3)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:57.964718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ef0c3-c0e0-43c4-b286-f803aea7dd73 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:57.966523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 563/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:03:57 2022] GET /identity/v3/projects/9ae4af84f05445aca485ecae61133bd3 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:03:57.976258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:03:58.081603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599df4-74e5-4ce1-a1b1-dcc1315d850b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:03:58.117469 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:03:58.118122 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:03:58.118402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:03:58.118681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:03:58.119542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=9ae4af84f05445aca485ecae61133bd3, user.id=c52d323cd4064a21ac2845a2f4a09635) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:03:58.119854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:03:58.121376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac72f-3844-4282-bf6d-984e4265b7e6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:03:58.127108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 583/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:03:57 2022] GET /identity/v3/role_assignments?user.id=c52d323cd4064a21ac2845a2f4a09635&scope.project.id=9ae4af84f05445aca485ecae61133bd3&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:00.309520 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafb3d1-5585-4546-af58-79d7df3cce59 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:00.310042 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafb3d1-5585-4546-af58-79d7df3cce59 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:00.310165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafb3d1-5585-4546-af58-79d7df3cce59 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:00.311572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 564/1147] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:00.317325 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:00.317737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:00.317878 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:00.336993 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:00.338235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:00.362270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:00.408615 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 584/1148] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:00.431276 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafb3d1-5585-4546-af58-79d7df3cce59 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:00.541088 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafb3d1-5585-4546-af58-79d7df3cce59 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:00.576213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:00.576989 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:00.577273 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:00.577565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:00.588355 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:04:00.596414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:04:00.597702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:00.598941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:04:00.601432 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 565/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:04:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:04:00.603598 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:00.734484 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9d5fc-ab5b-4124-afd0-a77ebd48a627 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:00.770757 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:00.771423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:00.771710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:00.771985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:00.772740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:00.773050 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:00.774499 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:00.783794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 585/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:04:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:00.790915 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:00.818546 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:00.901950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadefb6-1ce8-4f3a-814b-241904ad8874 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:00.939567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:00.940842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:00.941215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:00.941612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] PATH_INFO: `/v3/users/c52d323cd4064a21ac2845a2f4a09635` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:00.943758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] RBAC: Authorizing `identity:get_user(user_id=c52d323cd4064a21ac2845a2f4a09635)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:00.946123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:00.948695 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 566/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:04:00 2022] GET /identity/v3/users/c52d323cd4064a21ac2845a2f4a09635 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:00.955485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:00.982535 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:01.073936 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912854c7-13c8-4f54-bda6-0512215bec1b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:01.111010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e890aeb-271f-4da1-b779-f5211b7b776a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:01.111885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e890aeb-271f-4da1-b779-f5211b7b776a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:01.112172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e890aeb-271f-4da1-b779-f5211b7b776a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:01.112448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e890aeb-271f-4da1-b779-f5211b7b776a None admin] PATH_INFO: `/v3/projects/9ae4af84f05445aca485ecae61133bd3` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:01.113746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e890aeb-271f-4da1-b779-f5211b7b776a None admin] RBAC: Authorizing `identity:get_project(project_id=9ae4af84f05445aca485ecae61133bd3)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:01.115214 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e890aeb-271f-4da1-b779-f5211b7b776a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:01.116943 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 586/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:04:00 2022] GET /identity/v3/projects/9ae4af84f05445aca485ecae61133bd3 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:01.126033 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:01.231711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:01.242936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e320-8d8d-4a57-9aa0-13ba4fb199ad None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:01.279552 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bf73b-f92d-48c4-8c77-2a2146452ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:01.280476 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf73b-f92d-48c4-8c77-2a2146452ad9 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:01.280763 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf73b-f92d-48c4-8c77-2a2146452ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:01.281056 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf73b-f92d-48c4-8c77-2a2146452ad9 None admin] PATH_INFO: `/v3/projects/9ae4af84f05445aca485ecae61133bd3/users/c52d323cd4064a21ac2845a2f4a09635/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:01.283223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bf73b-f92d-48c4-8c77-2a2146452ad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ae4af84f05445aca485ecae61133bd3, user_id=c52d323cd4064a21ac2845a2f4a09635, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:01.284690 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bf73b-f92d-48c4-8c77-2a2146452ad9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:01.301747 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 567/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:04:01 2022] PUT /identity/v3/projects/9ae4af84f05445aca485ecae61133bd3/users/c52d323cd4064a21ac2845a2f4a09635/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:04:03.486710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b037e1-5b68-40b9-9e35-7d2e273f2647 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:03.487653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b037e1-5b68-40b9-9e35-7d2e273f2647 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:03.487653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b037e1-5b68-40b9-9e35-7d2e273f2647 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:03.489940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 587/1154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:03.496337 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:03.497124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:03.497349 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:03.517569 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:03.518783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:03.590086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 568/1155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:03.613979 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b037e1-5b68-40b9-9e35-7d2e273f2647 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:03.724423 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:03.739000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b037e1-5b68-40b9-9e35-7d2e273f2647 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:03.776419 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:03.777222 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:03.777543 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:03.777822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:03.790746 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:04:03.798378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:04:03.799771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:04:03.801584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 588/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:04:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:04:03.808808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:03.894525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:03.914838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c621cd-a99b-417c-ac72-c1e89310c47e None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:03.958136 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:03.958808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:03.959093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:03.959462 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:03.960221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:03.960534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:03.961961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:03.970484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 569/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:04:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:03.979402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:04.086051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5a366-0c05-4013-add8-b11643bfa5e9 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:04.123778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:04.125055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:04.125438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:04.125794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] PATH_INFO: `/v3/users/c52d323cd4064a21ac2845a2f4a09635` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:04.127794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] RBAC: Authorizing `identity:get_user(user_id=c52d323cd4064a21ac2845a2f4a09635)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:04.130256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:04.132736 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 589/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:04:03 2022] GET /identity/v3/users/c52d323cd4064a21ac2845a2f4a09635 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:04.142159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:04.253598 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6302-6976-49e3-b5c7-faa9a6c954a1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:04.294750 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:04.295410 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:04.295701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:04.295983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:04.308941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:04.373117 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:04.375486 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:04.483814 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 570/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:04:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:04:04.490017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:04.508785 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:04.597074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b569c-7ba0-4015-8602-4bef231b4879 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:04.634470 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6113a-dfcb-499d-927c-af5a9612f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:04.635333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6113a-dfcb-499d-927c-af5a9612f764 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:04.635614 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6113a-dfcb-499d-927c-af5a9612f764 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:04.635889 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6113a-dfcb-499d-927c-af5a9612f764 None admin] PATH_INFO: `/v3/projects/9ae4af84f05445aca485ecae61133bd3` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:04.637137 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6113a-dfcb-499d-927c-af5a9612f764 None admin] RBAC: Authorizing `identity:get_project(project_id=9ae4af84f05445aca485ecae61133bd3)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:04.638732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6113a-dfcb-499d-927c-af5a9612f764 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:04.640452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 590/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:04:04 2022] GET /identity/v3/projects/9ae4af84f05445aca485ecae61133bd3 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:04.647458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:04.695240 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:04.756694 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d8092-00ae-4756-b0cd-80676f7fd861 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:04.792902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:04.793511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:04.793791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:04.794063 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:04.794928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=9ae4af84f05445aca485ecae61133bd3, user.id=c52d323cd4064a21ac2845a2f4a09635) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:04.795471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:04.798039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe2ca-a930-43e6-8cdd-e293008beab6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:04.806159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 571/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:04:04 2022] GET /identity/v3/role_assignments?user.id=c52d323cd4064a21ac2845a2f4a09635&scope.project.id=9ae4af84f05445aca485ecae61133bd3&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:07.039570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099fa8-eed3-4a73-8bbb-ec0bce74d53a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:07.040159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099fa8-eed3-4a73-8bbb-ec0bce74d53a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:07.040159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099fa8-eed3-4a73-8bbb-ec0bce74d53a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:07.041579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 591/1162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:07.050617 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:07.051440 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:07.051687 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:07.073969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:07.075159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:07.149642 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 572/1163] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:07.174735 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099fa8-eed3-4a73-8bbb-ec0bce74d53a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:07.277331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:07.297690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099fa8-eed3-4a73-8bbb-ec0bce74d53a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:07.334154 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da202-0d07-4df3-a824-326b41462732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:07.334787 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da202-0d07-4df3-a824-326b41462732 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:07.335066 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da202-0d07-4df3-a824-326b41462732 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:07.335342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da202-0d07-4df3-a824-326b41462732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:07.404484 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472da202-0d07-4df3-a824-326b41462732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:07.405992 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472da202-0d07-4df3-a824-326b41462732 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:07.497048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:07.514696 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 592/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:04:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:04:07.521526 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:07.557030 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:07.622138 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53bd24-f60a-4eb9-9961-bd1e050f0edc None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:07.658712 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:07.659521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:07.659803 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:07.660073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:07.668465 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:07.670046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-1566a5fb-b1ad-41d5-a3a8-8f0cc8ac2d35 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:04:07.685278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:04:07.686729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 15:04:07.687988 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:04:07.689423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 573/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:04:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:04:07.692517 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da202-0d07-4df3-a824-326b41462732 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:07.803246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da202-0d07-4df3-a824-326b41462732 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:07.844782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:07.845447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:07.845732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:07.846009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:07.846742 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:07.847055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:07.848450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3ffdf-5783-4989-a7fb-1d625beba097 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:07.856126 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 593/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 19 15:04:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:10.070283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b2039-1eeb-4d52-8d85-ed4c4e92ce61 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:10.070850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b2039-1eeb-4d52-8d85-ed4c4e92ce61 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:10.070850 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b2039-1eeb-4d52-8d85-ed4c4e92ce61 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:10.072987 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 574/1167] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:10.081378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a4cdb-0324-4c0a-a254-4885134b39d6 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:10.081885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a4cdb-0324-4c0a-a254-4885134b39d6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:10.081997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a4cdb-0324-4c0a-a254-4885134b39d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:10.100317 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-a82a4cdb-0324-4c0a-a254-4885134b39d6 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:10.101583 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-a82a4cdb-0324-4c0a-a254-4885134b39d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:10.148404 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:10.186282 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 594/1168] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:04:10.219995 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b2039-1eeb-4d52-8d85-ed4c4e92ce61 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:10.326639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:10.335730 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b2039-1eeb-4d52-8d85-ed4c4e92ce61 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:10.372693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf14e6-a00f-4a3b-9f68-5100708a7e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:10.373389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf14e6-a00f-4a3b-9f68-5100708a7e7e None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:10.373675 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf14e6-a00f-4a3b-9f68-5100708a7e7e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:10.373951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf14e6-a00f-4a3b-9f68-5100708a7e7e None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:10.374716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf14e6-a00f-4a3b-9f68-5100708a7e7e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:10.376116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf14e6-a00f-4a3b-9f68-5100708a7e7e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:10.393837 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 575/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Thu May 19 15:04:10 2022] POST /identity/v3/domains => generated 267 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:04:12.647795 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8d73-b32e-4a0b-960e-f5e2095f9b9c None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:12.648388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8d73-b32e-4a0b-960e-f5e2095f9b9c None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:12.648388 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8d73-b32e-4a0b-960e-f5e2095f9b9c None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:12.650547 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 595/1170] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:12.659492 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:12.659928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:12.660079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:12.682885 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:12.684269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:12.756831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 576/1171] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:12.781364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8d73-b32e-4a0b-960e-f5e2095f9b9c None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:12.890363 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8d73-b32e-4a0b-960e-f5e2095f9b9c None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:12.926641 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4b18-f88f-4419-bda9-fd87d4f647d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:12.927444 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4b18-f88f-4419-bda9-fd87d4f647d7 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:12.927726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4b18-f88f-4419-bda9-fd87d4f647d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:12.927997 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4b18-f88f-4419-bda9-fd87d4f647d7 None admin] PATH_INFO: `/v3/domains/f203c162c4444febb8ea28a0da57f011` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:12.938982 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc4b18-f88f-4419-bda9-fd87d4f647d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f203c162c4444febb8ea28a0da57f011)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:12.940450 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc4b18-f88f-4419-bda9-fd87d4f647d7 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:12.942152 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 596/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 15:04:12 2022] GET /identity/v3/domains/f203c162c4444febb8ea28a0da57f011 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:12.948386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:13.048610 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:13.067914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b5a74-a748-43f3-be0d-c08957072530 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:13.106221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc174e75-37bb-4697-8136-921d9d6329a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:13.106887 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc174e75-37bb-4697-8136-921d9d6329a6 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:13.107169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc174e75-37bb-4697-8136-921d9d6329a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:13.107448 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc174e75-37bb-4697-8136-921d9d6329a6 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:13.108161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc174e75-37bb-4697-8136-921d9d6329a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:13.109652 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc174e75-37bb-4697-8136-921d9d6329a6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:13.138905 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 577/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:04:12 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:04:15.399519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb7956-3e46-49b9-908f-1e239dc9c82b None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:15.400376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb7956-3e46-49b9-908f-1e239dc9c82b None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:15.400376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb7956-3e46-49b9-908f-1e239dc9c82b None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:15.402703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 597/1174] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:15.408259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:15.408693 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:15.408793 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:15.427787 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:15.428977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:15.502245 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 578/1175] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:15.538777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb7956-3e46-49b9-908f-1e239dc9c82b None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:15.590469 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:15.658842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb7956-3e46-49b9-908f-1e239dc9c82b None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:15.695874 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7d5e-b8d3-420d-8821-ce80ad5541cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:15.696744 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7d5e-b8d3-420d-8821-ce80ad5541cb None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:15.697021 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7d5e-b8d3-420d-8821-ce80ad5541cb None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:15.697307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7d5e-b8d3-420d-8821-ce80ad5541cb None admin] PATH_INFO: `/v3/domains/f203c162c4444febb8ea28a0da57f011` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:15.698648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7d5e-b8d3-420d-8821-ce80ad5541cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=f203c162c4444febb8ea28a0da57f011)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:15.700123 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7d5e-b8d3-420d-8821-ce80ad5541cb None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:15.701885 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 598/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 19 15:04:15 2022] GET /identity/v3/domains/f203c162c4444febb8ea28a0da57f011 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:15.711114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:15.785555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:15.825391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313026a-2f0b-46c2-a195-c88ff94b4e3b None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:15.869871 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f37fa-3822-4cf1-b429-4de3358cc089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:15.870538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f37fa-3822-4cf1-b429-4de3358cc089 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:15.870838 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f37fa-3822-4cf1-b429-4de3358cc089 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:15.871107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f37fa-3822-4cf1-b429-4de3358cc089 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:15.871819 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f37fa-3822-4cf1-b429-4de3358cc089 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:15.873276 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f37fa-3822-4cf1-b429-4de3358cc089 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:15.895450 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 579/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:04:15 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:04:18.157263 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88616fef-7ec9-4f6b-ba11-d9a9ab006bfd None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:18.157890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88616fef-7ec9-4f6b-ba11-d9a9ab006bfd None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:18.157890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88616fef-7ec9-4f6b-ba11-d9a9ab006bfd None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:18.159238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 599/1178] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:18.165810 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:18.166560 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:18.166731 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:18.185719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:18.186952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:18.259178 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 580/1179] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:18.283515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88616fef-7ec9-4f6b-ba11-d9a9ab006bfd None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:18.390410 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88616fef-7ec9-4f6b-ba11-d9a9ab006bfd None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:18.397883 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:18.429777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:18.430505 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:18.430794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:18.431073 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:18.442264 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:04:18.449884 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:04:18.451165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:04:18.452237 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 600/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:04:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:04:18.458424 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:18.516601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:18.562962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3925ac5-562a-45c7-ad07-85b755c80ce1 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:18.599450 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:18.600124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:18.600413 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:18.600692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:18.601497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:18.601807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:18.603199 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:18.612449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 581/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:18.621078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:18.752398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d77fd0-623e-45e1-acc2-e6e295d07a8b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:18.791235 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:18.792118 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:18.792405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:18.792681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] PATH_INFO: `/v3/users/3ad71a383a91420cb691e3cb8bc5ab52` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:18.808392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] RBAC: Authorizing `identity:get_user(user_id=3ad71a383a91420cb691e3cb8bc5ab52)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:18.809929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:18.811633 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 601/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:04:18 2022] GET /identity/v3/users/3ad71a383a91420cb691e3cb8bc5ab52 => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:18.820930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:18.936095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ef058-1728-4ba7-81e6-ac27be7d1d38 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:18.954434 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:18.974909 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:18.975565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:18.975849 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:18.976133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:19.046633 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:19.048016 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:19.157182 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 582/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:04:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:04:19.165829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:19.199549 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:19.282095 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50a77-f7d7-4675-9c20-515886e0d586 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:19.318460 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea63c4-b3ca-49da-8f1b-5cb516d666da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:19.319256 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea63c4-b3ca-49da-8f1b-5cb516d666da None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:19.319508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea63c4-b3ca-49da-8f1b-5cb516d666da None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:19.319808 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea63c4-b3ca-49da-8f1b-5cb516d666da None admin] PATH_INFO: `/v3/projects/087e8f329f88428d9b103bad59bc3c41` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:19.321158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea63c4-b3ca-49da-8f1b-5cb516d666da None admin] RBAC: Authorizing `identity:get_project(project_id=087e8f329f88428d9b103bad59bc3c41)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:19.322656 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea63c4-b3ca-49da-8f1b-5cb516d666da None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:19.324402 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 602/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:04:19 2022] GET /identity/v3/projects/087e8f329f88428d9b103bad59bc3c41 => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:19.331624 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:19.354182 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:19.441473 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28d134-5d8d-4166-9705-ca54aa1d6d10 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:19.477489 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:19.478133 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:19.478411 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:19.478692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:19.479520 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=087e8f329f88428d9b103bad59bc3c41, user.id=3ad71a383a91420cb691e3cb8bc5ab52) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:19.479831 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:19.481275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3adf7-34cd-454e-a475-4023efd414c6 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:19.487031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 583/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:04:19 2022] GET /identity/v3/role_assignments?user.id=3ad71a383a91420cb691e3cb8bc5ab52&scope.project.id=087e8f329f88428d9b103bad59bc3c41&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:21.670223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471639d9-ae63-44c4-ab6e-5a7617f9fbf0 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:21.671092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471639d9-ae63-44c4-ab6e-5a7617f9fbf0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:21.671092 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471639d9-ae63-44c4-ab6e-5a7617f9fbf0 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:21.673329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 603/1186] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:21.681554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:21.682006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:21.682131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:21.700951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:21.702180 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:21.774284 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 584/1187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:04:21.797472 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471639d9-ae63-44c4-ab6e-5a7617f9fbf0 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:21.899560 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:21.910541 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471639d9-ae63-44c4-ab6e-5a7617f9fbf0 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:21.952969 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:21.954215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:21.954661 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:21.955090 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:21.971250 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:04:21.978774 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:04:21.980011 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:04:21.981474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 604/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:04:21.986863 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:22.004931 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:22.103124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabda99d-7358-4e8d-a8b1-31a647b2b99f None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:22.138997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:22.139666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:22.139956 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:22.140233 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:22.140981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:22.141681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:22.143059 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:22.151277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 585/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:22.160420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:22.266669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe1c0-e4f5-4711-9d14-6f459ac5124a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:22.305186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:22.306169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:22.306451 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:22.306724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] PATH_INFO: `/v3/users/3ad71a383a91420cb691e3cb8bc5ab52` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:22.308087 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] RBAC: Authorizing `identity:get_user(user_id=3ad71a383a91420cb691e3cb8bc5ab52)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:22.309574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:22.311246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 605/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:04:22 2022] GET /identity/v3/users/3ad71a383a91420cb691e3cb8bc5ab52 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:22.319880 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:22.396105 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:22.434258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac5faa-68ac-43e4-af1b-0404d8c363c9 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:22.471692 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb487f-74d3-4ebf-8474-8ffb085663ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:22.472553 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb487f-74d3-4ebf-8474-8ffb085663ad None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:22.472837 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb487f-74d3-4ebf-8474-8ffb085663ad None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:22.473114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb487f-74d3-4ebf-8474-8ffb085663ad None admin] PATH_INFO: `/v3/projects/087e8f329f88428d9b103bad59bc3c41` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:22.474494 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb487f-74d3-4ebf-8474-8ffb085663ad None admin] RBAC: Authorizing `identity:get_project(project_id=087e8f329f88428d9b103bad59bc3c41)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:22.475941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb487f-74d3-4ebf-8474-8ffb085663ad None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:22.477718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 586/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:04:22 2022] GET /identity/v3/projects/087e8f329f88428d9b103bad59bc3c41 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:22.487741 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:22.515393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:22.600052 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa6ebe-323f-4a6a-a593-9fd0b9667d54 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:22.645601 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f41fef-5d67-425b-a611-bc774cfd4f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:22.646502 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f41fef-5d67-425b-a611-bc774cfd4f39 None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:22.646788 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f41fef-5d67-425b-a611-bc774cfd4f39 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:22.647063 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f41fef-5d67-425b-a611-bc774cfd4f39 None admin] PATH_INFO: `/v3/projects/087e8f329f88428d9b103bad59bc3c41/users/3ad71a383a91420cb691e3cb8bc5ab52/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:22.649359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f41fef-5d67-425b-a611-bc774cfd4f39 None admin] RBAC: Authorizing `identity:create_grant(project_id=087e8f329f88428d9b103bad59bc3c41, user_id=3ad71a383a91420cb691e3cb8bc5ab52, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:22.650823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f41fef-5d67-425b-a611-bc774cfd4f39 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:22.666314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 606/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:04:22 2022] PUT /identity/v3/projects/087e8f329f88428d9b103bad59bc3c41/users/3ad71a383a91420cb691e3cb8bc5ab52/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:04:24.836948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b7e8f-9117-4ae8-bb6d-f58e8716dc16 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:24.837611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b7e8f-9117-4ae8-bb6d-f58e8716dc16 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:24.837611 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b7e8f-9117-4ae8-bb6d-f58e8716dc16 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:24.838975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 587/1193] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:04:24.845548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:24.846328 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:24.846553 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:24.868574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:04:24.869835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:04:24.943758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 607/1194] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:04:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:04:24.970389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b7e8f-9117-4ae8-bb6d-f58e8716dc16 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:25.088431 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b7e8f-9117-4ae8-bb6d-f58e8716dc16 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:25.128121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:25.128914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:25.129204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:25.129518 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:25.138120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:25.142716 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:04:25.150891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:04:25.152184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:04:25.153221 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 588/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:04:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:04:25.159082 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:25.177172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:25.273619 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d9591-2fb3-49eb-8d9a-77666c848f4a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:25.310504 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:25.311169 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:25.311449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:25.311720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:25.312467 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:25.312771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:25.314184 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:25.322425 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 608/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:04:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:25.331500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:25.442976 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8de3-e380-4568-a523-75fa5c51b6a0 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:25.478322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:25.479195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:25.479478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:25.479752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] PATH_INFO: `/v3/users/3ad71a383a91420cb691e3cb8bc5ab52` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:25.481090 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] RBAC: Authorizing `identity:get_user(user_id=3ad71a383a91420cb691e3cb8bc5ab52)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:25.482622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:25.484270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 589/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 19 15:04:25 2022] GET /identity/v3/users/3ad71a383a91420cb691e3cb8bc5ab52 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:25.490231 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:25.582251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:25.614100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1d75e-4832-4fda-a382-c2d8f4147ace None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:25.649642 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:25.650290 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:25.650570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:25.650852 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:25.721374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:25.722776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:25.793025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:25.824739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 609/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:04:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:04:25.831098 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:25.908123 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:25.943217 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e46e1-160a-4c01-a320-cf57daa9e453 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:25.979300 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27e6d0-e724-424f-b373-eadd192f3cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:25.980103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27e6d0-e724-424f-b373-eadd192f3cac None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:25.980391 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27e6d0-e724-424f-b373-eadd192f3cac None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:25.980673 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27e6d0-e724-424f-b373-eadd192f3cac None admin] PATH_INFO: `/v3/projects/087e8f329f88428d9b103bad59bc3c41` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:25.982075 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27e6d0-e724-424f-b373-eadd192f3cac None admin] RBAC: Authorizing `identity:get_project(project_id=087e8f329f88428d9b103bad59bc3c41)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:25.983536 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27e6d0-e724-424f-b373-eadd192f3cac None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:25.985259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 590/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 19 15:04:25 2022] GET /identity/v3/projects/087e8f329f88428d9b103bad59bc3c41 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:25.992158 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:04:26.103650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e05d6-ec88-48eb-bfa7-05fd27a04e93 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:04:26.151929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:04:26.152809 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:04:26.153148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:04:26.153528 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:04:26.154577 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=087e8f329f88428d9b103bad59bc3c41, user.id=3ad71a383a91420cb691e3cb8bc5ab52) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:04:26.154888 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:04:26.156358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d79bf1-77ee-49a0-ab3f-24eef4489991 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:04:26.162768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 610/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:04:25 2022] GET /identity/v3/role_assignments?user.id=3ad71a383a91420cb691e3cb8bc5ab52&scope.project.id=087e8f329f88428d9b103bad59bc3c41&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:04:35.974165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=32 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:04:36.159724 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=82 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:24.204964 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9332cf2-830c-4a2e-adea-217ee556f6fc None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:24.205650 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9332cf2-830c-4a2e-adea-217ee556f6fc None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:24.205650 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9332cf2-830c-4a2e-adea-217ee556f6fc None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:24.206914 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 591/1201] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:24.212574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:24.213046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:24.213146 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:24.230904 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:24.232223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:24.314870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 611/1202] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:05:24.340032 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9332cf2-830c-4a2e-adea-217ee556f6fc None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:24.453555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9332cf2-830c-4a2e-adea-217ee556f6fc None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:24.495044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:24.495768 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:24.496048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:24.496319 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:24.506853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:24.508304 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:24.526321 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:24.527550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:24.528835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:24.528835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:24.528835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:24.528835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:24.528835 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 592/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:24.533648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:24.653493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96e3fa-fbb1-4353-ae90-88a3378e6451 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:24.688493 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:24.695541 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:24.697277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:24.697277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:24.697277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:24.697983 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:24.698215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:24.700013 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2f2f1-8a8d-4afd-834a-2209892e7a67 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:24.719533 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 612/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:24.720556 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:24.792326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:24.852035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8b33-43e5-43a3-a92b-1e1ace64af9a None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:24.900256 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea1d09-8f4f-42a3-a654-68b85bbb2bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:24.900986 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea1d09-8f4f-42a3-a654-68b85bbb2bd1 None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:24.901371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea1d09-8f4f-42a3-a654-68b85bbb2bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:24.901710 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea1d09-8f4f-42a3-a654-68b85bbb2bd1 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:24.902626 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea1d09-8f4f-42a3-a654-68b85bbb2bd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:24.904170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea1d09-8f4f-42a3-a654-68b85bbb2bd1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:24.929455 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 593/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 19 15:05:24 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:05:27.378777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0949b26-cf78-4e97-b56c-4abbcfbe3fe6 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:27.379406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0949b26-cf78-4e97-b56c-4abbcfbe3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:27.379406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0949b26-cf78-4e97-b56c-4abbcfbe3fe6 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:27.380923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 613/1206] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:27.386627 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:27.387167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:27.387463 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:27.407282 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:27.408499 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:27.489061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 594/1207] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:05:27.513763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0949b26-cf78-4e97-b56c-4abbcfbe3fe6 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:27.636461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0949b26-cf78-4e97-b56c-4abbcfbe3fe6 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:27.675155 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:27.676058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:27.676345 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:27.676648 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:27.689215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:05:27.697551 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:27.704102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:27.705805 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 614/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:27.706918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:27.753270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:27.829259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c136c-09ad-4282-abbe-dad823123dbb None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:27.870621 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:27.871412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:27.871756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:27.872060 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:27.872939 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:27.873385 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:27.874818 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:27.884548 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 595/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:27.891606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:27.955541 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:28.017649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecdf05-b7c1-441f-8705-937ceca1683b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:28.059049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:28.059872 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:28.060299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:28.060582 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:28.072918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:28.074620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:28.098592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:28.100044 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:28.101346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:28.101346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:28.101346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:28.101346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:28.101346 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 615/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:28.106388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:28.227271 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:28.230819 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb6ee2-79c7-4246-98eb-507e55185f79 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:28.275644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:28.276426 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:28.276752 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:28.277071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:28.277963 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:28.278312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:28.279797 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:28.294151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 596/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:28.299953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:28.415426 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a028a9-c76a-4e91-855e-21f01c82e68a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:28.417193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:28.455311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:28.456283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:28.456578 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:28.456864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:28.465413 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:28.467151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:05:28.483268 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:28.484753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:28.486025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 616/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:28.490557 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:28.606019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21184bd4-72dc-42d9-bfe0-7c8345712f62 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:28.646560 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:28.647549 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:28.647861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:28.648108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:28.648960 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:28.649312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:28.650759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:28.670680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 597/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 15:05:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:28.677358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:28.799608 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec4d3c-75fd-4561-845a-0711ae7c3592 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:28.839929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:28.840655 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:28.841003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:28.841370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:28.879821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:28.928039 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:28.929675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:29.047311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 617/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:05:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:05:29.054052 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:29.075834 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:29.173365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b833d-4351-413a-828b-59549f07cf95 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:29.223205 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:29.224023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:29.224313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:29.224634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:29.239371 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:29.240916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:29.262916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:29.264527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:29.266223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:29.266223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:29.266223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:29.266223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:29.266223 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 598/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:29.273900 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:29.350430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:29.398074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c07a4-ac17-446a-90fb-aeacfc777e5d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:29.439581 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:29.440284 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:29.440550 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:29.440819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:29.441687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:29.442097 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:29.443784 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:29.457297 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 618/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:29.463015 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:29.548384 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:29.588114 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86e5c-d8c9-488e-991d-fefdbf799c22 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:29.627514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:29.628249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:29.628525 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:29.628800 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:29.636211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:29.637719 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:05:29.652788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:29.654035 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:29.655124 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:29.660073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 599/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:29.661849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:29.784441 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e397b-0e29-4066-955a-f5c9ba11c069 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:29.810900 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:29.826248 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:29.827043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:29.827348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:29.827655 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:29.828481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:29.828819 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:29.830310 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d49c7-69ef-48fc-a4e7-3be67b9bae4a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:29.846737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 619/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:29.853193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:29.987259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063298-6062-49b3-bb3a-61969fa712c2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:30.030638 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:30.037942 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:30.038629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:30.038944 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:30.039291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:30.040180 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=624b80e56c8c44e8bc4c1b43089d50f6) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:30.040496 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:30.042034 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0141f4-f45f-42a4-bdf5-6fe9eef1a396 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:30.048973 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 600/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:05:29 2022] GET /identity/v3/role_assignments?user.id=624b80e56c8c44e8bc4c1b43089d50f6&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:32.339037 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a6f6a-9ee4-4766-a078-58cceefb2d46 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:32.339653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a6f6a-9ee4-4766-a078-58cceefb2d46 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:32.339653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a6f6a-9ee4-4766-a078-58cceefb2d46 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:32.341189 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 620/1220] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:32.346649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:32.347139 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:32.347254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:32.364336 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:32.365613 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:32.431942 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 601/1221] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:05:32.454631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a6f6a-9ee4-4766-a078-58cceefb2d46 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:32.566069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a6f6a-9ee4-4766-a078-58cceefb2d46 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:32.603223 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:32.604010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:32.604298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:32.604572 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:32.615120 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:05:32.622861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:32.624414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:05:32.625835 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:32.629483 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 621/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:05:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:32.630507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:32.738477 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2cfa5-f138-41d1-a496-1702a927c7e0 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:32.762783 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:32.777822 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:32.778688 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:32.778992 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:32.779272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:32.780103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:32.780412 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:32.781908 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:32.791623 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 602/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:05:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:32.797930 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:32.847081 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:32.917481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5806-83d8-4b17-83e2-46553abac528 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:32.953911 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:32.954721 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:32.955001 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:32.955274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:32.966216 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:32.967734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:32.985935 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:32.987474 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:32.988723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:32.988723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:32.988723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:32.988723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:32.988723 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 622/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:32.993941 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:33.103797 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7e564-7354-4d13-967f-f455c1bc8c9b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:33.137250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:33.138099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:33.138387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:33.138707 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:33.139522 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:33.139832 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:33.141820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:33.157031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 603/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:05:33.163331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:33.264342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:33.278519 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5646bf-ddc5-45e7-b4c9-09a2badb9f06 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:33.317837 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:33.318661 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:33.318940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:33.319210 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] PATH_INFO: `/v3/users/placement` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:33.326890 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:33.328352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:05:33.343923 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:33.345822 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:33.347331 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 623/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:33.351188 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:33.405287 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:33.466997 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c8581-131c-49fa-979a-f433d587bb55 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:33.500805 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:33.501506 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:33.501791 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:33.502075 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:33.502923 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:33.503237 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:33.504678 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:33.523450 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 604/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 15:05:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:33.529603 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:33.638371 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72544d8c-25e7-45dc-ab90-ad03fbbc95ec None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:33.676559 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:33.677316 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:33.677602 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:33.677878 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:33.688952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:33.691515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:33.709860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:33.711172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:33.712287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:33.712287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:33.712287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:33.712287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:33.712287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 624/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:33.717036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:33.832416 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d117696-ebe9-47ca-9bfb-1c5ad51994aa None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:33.835121 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:33.873120 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:33.873910 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:33.874201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:33.874484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:33.875210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:33.875578 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:33.877005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:33.891232 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 605/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:33.897924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:33.908532 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:34.004036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55761f1d-9c5d-4025-ad07-d46568e8b8ec None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:34.038537 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:34.039451 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:34.039771 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:34.040100 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:34.048393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:34.050096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:05:34.065333 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:34.066651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:34.067986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 625/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:05:34.072994 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:34.185821 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf897e-5c95-4e09-8f67-d989abc02a2f None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:34.222820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:34.223554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:34.223902 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:34.224224 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:34.225145 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:34.225488 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:34.227102 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b19c-c3c3-48a2-8e6d-a2fdde3a2bde None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:34.239961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 606/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:05:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:34.246737 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:34.324485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:34.363649 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4df9a-f4e7-474c-9e3c-f63c882dbc85 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:34.400860 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c7102-5cc5-4859-bfcb-1f0038365886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:34.401801 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c7102-5cc5-4859-bfcb-1f0038365886 None admin] REQUEST_METHOD: `PUT` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:34.402125 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c7102-5cc5-4859-bfcb-1f0038365886 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:34.402445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c7102-5cc5-4859-bfcb-1f0038365886 None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/624b80e56c8c44e8bc4c1b43089d50f6/roles/f548e28c81fd4132b0654cd1a3a1d975` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:34.417541 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c7102-5cc5-4859-bfcb-1f0038365886 None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=624b80e56c8c44e8bc4c1b43089d50f6, role_id=f548e28c81fd4132b0654cd1a3a1d975)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:34.419074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c7102-5cc5-4859-bfcb-1f0038365886 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:34.433925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 626/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:05:34 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/624b80e56c8c44e8bc4c1b43089d50f6/roles/f548e28c81fd4132b0654cd1a3a1d975 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:05:36.808621 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e3b6d-8c58-4fd9-a834-77368de0f0a2 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:36.809215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e3b6d-8c58-4fd9-a834-77368de0f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:36.809215 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e3b6d-8c58-4fd9-a834-77368de0f0a2 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:36.810624 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 607/1233] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:36.816003 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:36.816464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:36.816574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:36.834880 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:36.836311 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:36.914487 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 627/1234] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:05:36.938980 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e3b6d-8c58-4fd9-a834-77368de0f0a2 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:36.970272 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:37.054093 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e3b6d-8c58-4fd9-a834-77368de0f0a2 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:37.093310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:37.094046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:37.094326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:37.094600 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] PATH_INFO: `/v3/roles/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:37.104781 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:05:37.112962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:37.114204 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:37.115261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 608/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 19 15:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:37.120849 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:37.171363 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:37.234230 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac349c-e82a-4867-9199-cc77f696b440 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:37.274193 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:37.274393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:37.274653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:37.275000 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:37.275952 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] RBAC: Adding query filter params (name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:37.276383 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:37.277829 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:37.287616 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 628/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 19 15:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:05:37.293899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:37.386884 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:37.403732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e463a-1f89-4b79-b7bc-763c7ea3747b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:37.439812 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:37.440770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:37.441125 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:37.441478 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:37.454543 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:37.456516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:37.478073 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:37.479501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:37.480653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:37.480653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:37.480653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:37.480653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:37.480653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 609/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:37.485584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:37.602109 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991da75a-291c-4c86-9c51-a1e208da5894 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:37.612314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:37.639828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:37.640440 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:37.641084 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:37.641429 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:37.642161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:37.642471 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:37.643889 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:37.657406 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 629/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:37.663477 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:37.776718 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e04d1-81e1-4aa2-9285-04136bbab550 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:37.814334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:37.815288 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:37.815657 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:37.815998 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:37.818961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:37.824711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:37.826565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:05:37.843039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:37.844423 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:37.845625 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 610/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:37.850521 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:37.971559 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d68f2-ffd1-4c21-bfcb-9cf582481d80 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:38.008755 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:38.009463 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:38.009745 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:38.010019 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:38.011027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:38.011438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:38.013145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:38.039447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 630/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 15:05:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:38.040789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:38.151805 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e2b9f-e17a-429c-8fb4-5da53954aa72 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:38.189421 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:38.190078 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:38.190365 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:38.190641 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:38.263397 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:38.276666 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:38.278417 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:38.391514 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 611/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:05:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:05:38.398739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:38.430789 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:38.514041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10525529-5e10-4f33-ba21-403ef4b7380a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:38.548503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:38.549342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:38.549630 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:38.549906 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:38.560199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:38.561726 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:38.579359 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:38.580749 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:38.582036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:38.582036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:38.582036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:38.582036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:38.582036 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 631/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:38.586965 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:38.693582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:38.700449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508fa95-2d7a-4509-b729-3f0f0ff98a63 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:38.738177 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:38.738879 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:38.739186 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:38.739470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:38.740242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:38.740607 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:38.742045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:38.759030 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 612/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:38.765395 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:38.858197 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:38.885699 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9d4d6-b198-486f-84ef-36ea22e0b57d None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:38.929708 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:38.930709 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:38.931031 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:38.931307 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:38.940703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:38.942549 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:05:38.959409 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:38.960864 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:38.962202 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 632/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:05:38.967163 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:39.097415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad91bb-f029-40f2-b954-72807ac5b459 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:39.138180 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:39.138890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:39.139208 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:39.139459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:39.140270 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:39.140585 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:39.142444 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9e83e-c08a-4cdb-a752-a995881e45f9 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:39.155521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 613/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:39.162167 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:39.279447 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c06540-9614-4d32-9c42-51796f28be20 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:39.320690 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:39.321393 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:39.321681 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:39.321954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:39.322794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] RBAC: Adding query filter params (role.id=f548e28c81fd4132b0654cd1a3a1d975, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=624b80e56c8c44e8bc4c1b43089d50f6) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:39.323108 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:39.324582 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcb43f-6220-4a13-a195-2996de517e70 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:39.330832 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 633/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:05:39 2022] GET /identity/v3/role_assignments?user.id=624b80e56c8c44e8bc4c1b43089d50f6&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=f548e28c81fd4132b0654cd1a3a1d975 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:41.837519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302cc1b-efbc-44dd-9e6d-0da7d4d90439 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:41.838127 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302cc1b-efbc-44dd-9e6d-0da7d4d90439 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:41.838242 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302cc1b-efbc-44dd-9e6d-0da7d4d90439 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:41.839928 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 614/1247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:41.848110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:41.848739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:41.848739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:41.854314 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:41.871171 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:41.872770 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:41.961687 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 634/1248] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:05:41.988419 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302cc1b-efbc-44dd-9e6d-0da7d4d90439 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:42.008930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:42.118275 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302cc1b-efbc-44dd-9e6d-0da7d4d90439 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:42.157104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:42.158222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:42.158516 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:42.158795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:42.171100 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:05:42.179742 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:42.181077 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:42.182334 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 615/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:05:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:42.187893 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:42.313435 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c39e4-46b7-4efb-a343-c1cc3ff3ebeb None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:42.348738 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:42.355033 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:42.355953 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:42.356242 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:42.356545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:42.357414 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:42.357720 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:42.359133 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:42.368176 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 635/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:05:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:42.374771 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:42.467292 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:42.496428 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1f847-78f7-48c6-942d-4e37fe997c94 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:42.534030 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:42.534896 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:42.535184 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:42.535463 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:42.547039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:42.548586 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:42.566150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:42.567570 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:42.569036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:42.569036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:42.569036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:42.569036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:42.569036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 616/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:42.573326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:42.682716 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdacea-3aea-4bb3-b48c-28deb4b1e5ce None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:42.719430 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:42.720094 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:42.720379 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:42.720684 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:42.721448 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:42.721798 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:42.723150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:42.737272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 636/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:42.742388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:42.854601 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d326e71-3f26-4073-9831-1dba2f5b6cbc None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:42.887383 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:42.896962 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:42.897843 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:42.898109 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:42.898406 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] PATH_INFO: `/v3/users/placement` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:42.908031 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:42.909904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:05:42.928795 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:42.930290 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:42.931864 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 617/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:42.936761 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:43.005449 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:43.067877 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae887d8-fbbb-4631-8434-308c885c4948 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:43.104365 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:43.105056 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:43.105369 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:43.105650 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:43.106672 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:43.106976 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:43.108442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:43.131017 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 637/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 15:05:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:43.137266 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:43.264572 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851544-0246-4c88-8fe5-f5589b0c00da None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:43.314222 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:43.314877 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:43.315167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:43.315446 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:43.391957 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:43.409214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:43.411362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:43.526971 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 618/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:05:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:05:43.527945 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:43.634304 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eea51-c83f-43e1-82e0-45507657cd68 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:43.648364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:43.670147 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:43.671085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:43.671465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:43.671850 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:43.682891 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:43.684539 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:43.702842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:43.704186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:43.705670 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:43.705670 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:43.705670 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:43.705670 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:43.705670 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 638/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:05:43.710079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:43.811770 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:43.825308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edd71b-6228-40d2-b5b0-92203403f0ce None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:43.862111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:43.863230 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:43.863701 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:43.864166 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:43.865254 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:43.865777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:43.867495 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:43.883511 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 619/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:05:43.889703 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:43.999238 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7a03-6256-498b-8b1b-9cdb2a16eb37 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:44.037500 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:44.038342 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:44.038595 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:44.038901 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:44.045844 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:44.047322 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:05:44.060986 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:44.062606 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:44.063717 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 639/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:05:44.068559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:44.184736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3272c-874a-4b2b-815f-2afe868395f1 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:44.222907 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:44.223587 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:44.223873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:44.224151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] PATH_INFO: `/v3/projects` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:44.224966 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:44.225303 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:44.226788 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f381-3e5c-4a88-84a0-3660ab4423a1 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:44.240983 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 620/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:44.247580 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:44.281924 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:44.364192 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05640c-e2c0-4d9e-a52c-e6bbc0d8bb2a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:44.403922 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:44.404592 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:44.404879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:44.405144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:44.406022 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=624b80e56c8c44e8bc4c1b43089d50f6) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:44.406338 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:44.407847 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e6bab-770f-48da-8c11-5c013683649e None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:44.414494 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 640/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:05:44 2022] GET /identity/v3/role_assignments?user.id=624b80e56c8c44e8bc4c1b43089d50f6&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=8f859c3494964044956fd86bc077eaf0 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:46.673965 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a60b66-ad9d-4328-96bd-01c78d55a802 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:46.674886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a60b66-ad9d-4328-96bd-01c78d55a802 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:46.674886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a60b66-ad9d-4328-96bd-01c78d55a802 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:46.677064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 621/1261] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:46.683417 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:46.684133 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:46.684348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:46.708301 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:46.709539 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:46.788236 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 641/1262] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:05:46.811219 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a60b66-ad9d-4328-96bd-01c78d55a802 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:46.828190 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:46.940167 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a60b66-ad9d-4328-96bd-01c78d55a802 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:46.979420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:46.980229 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:46.980518 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:46.980799 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:46.991322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:05:46.999057 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:47.000374 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:47.001555 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 622/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:05:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:47.006594 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:47.027651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:47.111400 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce3f46-147c-41c0-b780-5fddee9bb79d None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:47.148837 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:47.149532 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:47.149820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:47.150096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] PATH_INFO: `/v3/roles` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:47.150854 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:47.151188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:47.152578 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:47.161754 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 642/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:47.168550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:47.242281 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:47.286741 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cf241-ca6f-4fe8-b9b9-6987a45d6797 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:47.328161 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:47.328969 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:47.329276 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:47.329571 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:47.341822 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:47.343299 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:47.360961 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:47.364047 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:47.365387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:47.365387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:47.365387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:47.365387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:47.365387 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 623/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:47.368867 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:47.466691 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:47.492767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649757f5-46ac-4847-856c-3da4f64d471b None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:47.530415 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:47.531074 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:47.531356 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:47.531634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:47.532354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:47.532665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:47.534091 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:47.545988 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 643/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:47.554906 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:47.666839 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df846b20-9ac9-4eec-b070-718151fac546 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:47.689244 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:47.706250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:47.707066 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:47.707358 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:47.707630 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:47.715729 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:47.717200 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:05:47.731922 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:47.733308 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:47.734756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 624/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:47.739196 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:47.846229 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2ff2b-67ad-4c5f-b566-f05632e380f6 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:47.886005 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:47.886962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:47.887285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:47.887664 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] PATH_INFO: `/v3/users` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:47.888776 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:47.889139 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:47.891213 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:47.903051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:47.912962 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 644/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 15:05:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:47.921107 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:48.028220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa150440-cb02-4727-9fef-ae391562b3b3 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:48.064048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:48.064841 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:48.065127 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:48.065437 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:48.076859 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:48.078346 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:48.095680 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:48.097046 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:48.098512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:48.098512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:48.098512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:48.098512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:48.098512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 625/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:05:48.103155 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:48.206219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b89c6-1ffe-432e-9a49-a07554587f31 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:48.242584 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:48.243625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:48.244078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:48.244518 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:48.245753 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:48.246259 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:48.248084 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:48.260281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 645/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:48.266677 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:48.301867 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:48.381150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164344e1-07e4-4922-9c5c-c0e30c5d6485 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:48.417695 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:48.418497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:48.418807 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:48.419094 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:48.426325 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:48.427797 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:05:48.444808 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:48.446074 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:48.447278 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 626/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:48.452341 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:48.512759 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:48.562731 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba208e-a818-4be8-83d7-71e3a38a253a None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:48.597823 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:48.598481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:48.598768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:48.599021 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:48.599833 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:48.600150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:48.601634 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9b927-59d9-4799-a89e-5c4cf4dd71ae None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:48.613826 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 646/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 15:05:48.620261 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:48.730464 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:48.742519 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bc751-256c-4a11-967d-f9cec026d557 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:48.783065 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cd93b-4f91-4875-9b35-9791992ee8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:48.783965 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cd93b-4f91-4875-9b35-9791992ee8f0 None admin] REQUEST_METHOD: `PUT` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:48.784260 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cd93b-4f91-4875-9b35-9791992ee8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:48.784542 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cd93b-4f91-4875-9b35-9791992ee8f0 None admin] PATH_INFO: `/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/624b80e56c8c44e8bc4c1b43089d50f6/roles/8f859c3494964044956fd86bc077eaf0` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:48.786845 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cd93b-4f91-4875-9b35-9791992ee8f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=369ac6c738f848fa8c426fb690d2db7f, user_id=624b80e56c8c44e8bc4c1b43089d50f6, role_id=8f859c3494964044956fd86bc077eaf0)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:48.788328 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cd93b-4f91-4875-9b35-9791992ee8f0 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:48.803855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 627/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 19 15:05:48 2022] PUT /identity/v3/projects/369ac6c738f848fa8c426fb690d2db7f/users/624b80e56c8c44e8bc4c1b43089d50f6/roles/8f859c3494964044956fd86bc077eaf0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 15:05:50.986209 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222e45-1962-4546-91d8-62a51d364000 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:50.987281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222e45-1962-4546-91d8-62a51d364000 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:50.987281 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222e45-1962-4546-91d8-62a51d364000 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:50.989326 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 647/1274] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:50.997426 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:50.997891 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:50.998013 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:51.017793 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:51.018956 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:51.087820 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 628/1275] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 15:05:51.118565 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222e45-1962-4546-91d8-62a51d364000 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:51.225145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222e45-1962-4546-91d8-62a51d364000 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:51.242734 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:51.263486 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:51.264283 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:51.264566 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:51.264840 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:51.275926 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 15:05:51.283545 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:51.284894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:51.286227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 648/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 19 15:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:51.292207 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:51.379501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:51.408823 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d0867-554a-4bfb-8f16-f51d11de11c2 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:51.445193 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:51.445898 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:51.446187 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:51.446457 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] PATH_INFO: `/v3/roles` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:51.447208 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:51.447512 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:51.448897 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:51.457331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 629/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 19 15:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:51.463166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:51.571747 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3d87f-dc60-435e-93cf-16f728137617 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:51.615454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:51.616251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:51.616544 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:51.616825 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:51.627646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:51.629119 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:51.646803 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:51.647990 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:51.649465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:51.649465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:51.649465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:51.649465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:51.649465 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 649/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 15:05:51.655311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:51.772565 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaa128-b0e2-4a15-8b39-72d7eca22041 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:51.810388 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:51.811045 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:51.811329 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:51.811576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] PATH_INFO: `/v3/domains` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:51.812341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:51.812647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:51.814064 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:51.827364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 630/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:51.838376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:51.887105 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:51.972242 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c330da-c22e-442c-ac0a-2c0344706463 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:52.007227 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:52.008039 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:52.008321 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:52.008596 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:52.016653 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:52.018160 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 15:05:52.032485 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:52.033763 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:52.034914 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 650/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:52.041955 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:52.046156 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:52.146711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c69288-4d04-41a2-ab72-960af09ce53b None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:52.182792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:52.183458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:52.183743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:52.184024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] PATH_INFO: `/v3/users` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:52.184916 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:52.185231 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:52.186702 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:52.203965 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 631/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 19 15:05:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:52.210625 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:52.310710 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:52.321285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200badb3-478c-48b6-b7f1-147b964a5b47 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:52.356686 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:52.357352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:52.357632 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:52.357902 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:52.427982 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:52.429408 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:52.521846 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:52.538040 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 651/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 19 15:05:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 15:05:52.544500 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:52.610853 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:52.654173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e81336-1760-4cf6-9660-5a1ab9c4f454 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:52.690248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:52.690973 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:52.691262 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:52.691540 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:52.702653 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:52.704116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 15:05:52.723250 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:52.724427 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:52.726458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 15:05:52.726458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 15:05:52.726458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 15:05:52.726458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:52.726458 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 632/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 19 15:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:52.732978 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:52.839455 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341aa390-137b-4b72-9591-47da8ca49d96 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:52.875199 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:52.875861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:52.876148 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:52.876381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:52.877152 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:52.877501 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:52.878894 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:52.891392 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 652/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 19 15:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:52.897951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:53.018109 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494094f5-dcad-44da-9df5-22a9d1f9bda2 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:53.032471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:53.061829 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:53.062766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:53.063048 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:53.063324 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:53.070331 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:53.071772 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 15:05:53.085554 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application with Lock( May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:53.086732 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: ERROR keystone.server.flask.application May 19 15:05:53.088218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 633/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 19 15:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:53.092861 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:53.145897 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:53.199798 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef8400-ea43-4be6-b4be-32aa75151ff0 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:53.235046 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:53.235693 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:53.235982 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:53.236246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] PATH_INFO: `/v3/projects` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:53.237035 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:53.237364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:53.238816 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1b42b-13fc-44c9-9e5d-dc6a16e63900 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:53.250828 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 653/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 19 15:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:53.258039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:53.368817 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013d2f7-fd5f-4061-bcf4-965817690608 None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:53.404798 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:53.405474 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:53.405760 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:53.406039 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:53.406890 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] RBAC: Adding query filter params (role.id=8f859c3494964044956fd86bc077eaf0, scope.project.id=369ac6c738f848fa8c426fb690d2db7f, user.id=624b80e56c8c44e8bc4c1b43089d50f6) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:53.407208 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:53.408655 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee5afd-a05c-4676-94ff-aa0b62c695ff None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:53.416756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 634/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 19 15:05:53 2022] GET /identity/v3/role_assignments?user.id=624b80e56c8c44e8bc4c1b43089d50f6&scope.project.id=369ac6c738f848fa8c426fb690d2db7f&role.id=8f859c3494964044956fd86bc077eaf0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:55.655043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775ce3a-7b5f-4960-bd7c-f016d20ae94f None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:55.655631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775ce3a-7b5f-4960-bd7c-f016d20ae94f None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:55.655631 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775ce3a-7b5f-4960-bd7c-f016d20ae94f None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:55.657766 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 654/1288] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:55.666577 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:55.667009 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:55.667170 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:55.686191 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:55.687415 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:55.722855 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:55.773977 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 635/1289] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:05:55.798303 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775ce3a-7b5f-4960-bd7c-f016d20ae94f None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:55.923579 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775ce3a-7b5f-4960-bd7c-f016d20ae94f None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:55.937620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:55.964838 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:55.965778 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:55.966048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:55.966361 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:55.966995 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:55.968412 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:05:55.980599 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:05:55.982030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:05:55.983069 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 655/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:05:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:05:55.988216 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:56.092876 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f39c59-0c9f-413f-9c70-df5ee7dacd3f None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:56.128062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:56.128711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:56.128994 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:56.129277 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:56.129999 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:56.130306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:56.131681 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd089-7fb6-41b1-be93-7ff490e63cc4 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:56.136703 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 636/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 15:05:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:56.143071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:56.260651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f28421-66fb-45e0-bb9d-84f075e0b957 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:56.297172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:56.297874 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:56.298166 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:56.298442 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] PATH_INFO: `/v3/services` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:56.299164 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] RBAC: Adding query filter params (type=placement) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:05:56.299475 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:56.300882 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad601-2acd-4731-b6af-91e8605b5028 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:56.305671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 656/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 15:05:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:05:58.508289 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3a70-ae7b-4836-9045-6c00857b2aa8 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:58.508886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3a70-ae7b-4836-9045-6c00857b2aa8 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:58.508886 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3a70-ae7b-4836-9045-6c00857b2aa8 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:58.511176 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 637/1293] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:05:58.517814 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3396-bbbb-48a6-a735-7d621eeec7d4 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:58.518218 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3396-bbbb-48a6-a735-7d621eeec7d4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:58.518376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3396-bbbb-48a6-a735-7d621eeec7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:58.538186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-cfdb3396-bbbb-48a6-a735-7d621eeec7d4 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:05:58.539357 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-cfdb3396-bbbb-48a6-a735-7d621eeec7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:05:58.573018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:58.616353 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 657/1294] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:05:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:05:58.640051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3a70-ae7b-4836-9045-6c00857b2aa8 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:05:58.675233 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:05:58.749259 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3a70-ae7b-4836-9045-6c00857b2aa8 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:05:58.785061 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192690-5ace-4e83-9f61-f071a28738ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:05:58.785756 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192690-5ace-4e83-9f61-f071a28738ab None admin] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:05:58.786044 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192690-5ace-4e83-9f61-f071a28738ab None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:05:58.786322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192690-5ace-4e83-9f61-f071a28738ab None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:05:58.787049 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192690-5ace-4e83-9f61-f071a28738ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:05:58.788505 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192690-5ace-4e83-9f61-f071a28738ab None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:05:58.805169 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 638/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 19 15:05:58 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:06:01.039358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8ebbe-3c3f-47b3-8dd4-0f702ffc95b4 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:01.040085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8ebbe-3c3f-47b3-8dd4-0f702ffc95b4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:01.040085 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8ebbe-3c3f-47b3-8dd4-0f702ffc95b4 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:01.041364 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 658/1296] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:06:01.047970 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:01.048728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:01.048947 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:01.071024 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:06:01.072232 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:06:01.151743 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 639/1297] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:06:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:06:01.187456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8ebbe-3c3f-47b3-8dd4-0f702ffc95b4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:01.300398 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8ebbe-3c3f-47b3-8dd4-0f702ffc95b4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:01.336671 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:01.337507 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:01.337791 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:01.338071 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:01.338739 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:01.340151 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:06:01.360675 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:06:01.362051 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:06:01.363270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 659/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:06:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:06:01.368274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:01.388332 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:06:01.483326 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bfc02-8c92-42c7-9f08-b006baf8bc26 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:01.529258 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:01.529935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:01.530224 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:01.530507 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:01.531231 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] RBAC: Adding query filter params (name=placement) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:06:01.531548 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:01.532935 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:01.538632 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 640/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 15:06:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:06:01.545055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:01.583064 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:06:01.654180 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e47a6a-b6b9-48f8-99f5-857878f4b835 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:01.697508 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:01.698162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:01.698445 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:01.698718 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:01.699513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] RBAC: Adding query filter params (interface=public, service_id=fa703571add34348a3ae3c6b37477eed, region_id=RegionOne) {{(pid=277131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:06:01.699821 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:01.701217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7523a-548e-4ddb-bc4b-4f0ca2a663a2 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:01.707005 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 660/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 19 15:06:01 2022] GET /identity/v3/endpoints?service_id=fa703571add34348a3ae3c6b37477eed&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:06:01.713071 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:01.802471 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:06:01.825150 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953b6e3-2dc4-4b45-91bf-4b17aed40b1e None admin] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:01.862305 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0050f-9d78-4e5d-a8b4-1f4acc216cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:01.863019 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0050f-9d78-4e5d-a8b4-1f4acc216cea None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:01.863306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0050f-9d78-4e5d-a8b4-1f4acc216cea None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:01.863614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0050f-9d78-4e5d-a8b4-1f4acc216cea None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:01.864466 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0050f-9d78-4e5d-a8b4-1f4acc216cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:01.866103 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0050f-9d78-4e5d-a8b4-1f4acc216cea None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:01.874523 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 641/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 19 15:06:01 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 15:06:04.060354 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c3b7d-7e18-4e02-9529-f63e5201b0b0 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:04.061429 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c3b7d-7e18-4e02-9529-f63e5201b0b0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:04.061429 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c3b7d-7e18-4e02-9529-f63e5201b0b0 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:04.062730 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 661/1302] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:06:04.069306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:04.070030 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:04.070214 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:04.095982 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:06:04.097213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:06:04.168386 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 642/1303] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:06:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 15:06:04.191799 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c3b7d-7e18-4e02-9529-f63e5201b0b0 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:04.298078 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:06:04.305900 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c3b7d-7e18-4e02-9529-f63e5201b0b0 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:04.341434 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:04.342138 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:04.342421 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:04.342696 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:04.343358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:04.344801 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 15:06:04.357043 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 15:06:04.358390 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.get_or_create( May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application with Lock( May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self._enter() May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application return self.creator() May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application created_value = creator( May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: ERROR keystone.server.flask.application May 19 15:06:04.359546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 662/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 19 15:06:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 15:06:04.364846 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:04.443550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:06:04.472247 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d03a00-d988-498f-8b8d-27efc33ea1d2 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:04.507195 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:04.507868 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:04.508153 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:04.508430 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] PATH_INFO: `/v3/services` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:04.509144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 15:06:04.509484 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:04.510874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f418f-a3a9-4739-b395-e2ebe316a4d2 None admin] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:04.517231 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 643/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 19 15:06:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 15:06:04.523329 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:06:04.624350 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e6fa-cd08-4e07-a621-d136405d1759 None admin] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:06:04.659980 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807e0e0-a677-41b1-9efb-d0b821fdd3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7ff97c0da314994a8835ccf5732ab5e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:06:04.660635 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e0e0-a677-41b1-9efb-d0b821fdd3a5 None admin] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:06:04.660915 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e0e0-a677-41b1-9efb-d0b821fdd3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:06:04.661190 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e0e0-a677-41b1-9efb-d0b821fdd3a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:06:04.661964 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807e0e0-a677-41b1-9efb-d0b821fdd3a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:06:04.663370 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807e0e0-a677-41b1-9efb-d0b821fdd3a5 None admin] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:06:04.687446 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 663/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 19 15:06:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 15:06:14.513165 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=30 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:06:14.681009 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:11.064475 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789e00-1d59-48ec-9719-2d27cddc60db None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:11.065111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789e00-1d59-48ec-9719-2d27cddc60db None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:11.065111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789e00-1d59-48ec-9719-2d27cddc60db None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:11.066534 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 644/1307] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:11.072665 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f672f1-55cc-4303-9944-abeaec57d516 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:11.073258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f672f1-55cc-4303-9944-abeaec57d516 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:11.073258 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f672f1-55cc-4303-9944-abeaec57d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:11.206620 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-89f672f1-55cc-4303-9944-abeaec57d516 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:11.208378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-89f672f1-55cc-4303-9944-abeaec57d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:11.360712 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 664/1308] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:11.423540 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c102f-3cef-449a-b2ad-c3294e3c4c79 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:11.423842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c102f-3cef-449a-b2ad-c3294e3c4c79 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:11.424104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c102f-3cef-449a-b2ad-c3294e3c4c79 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:11.425801 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 645/1309] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 15:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:11.436515 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627e399-641c-45c3-9480-09d1bcc21e45 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:11.437037 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627e399-641c-45c3-9480-09d1bcc21e45 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:11.437420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627e399-641c-45c3-9480-09d1bcc21e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:11.525925 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-a627e399-641c-45c3-9480-09d1bcc21e45 None None] MFA Rules not processed for user `abe824c999d34757b0c84e96bfa3d1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:11.527299 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-a627e399-641c-45c3-9480-09d1bcc21e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:11.550246 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:11.628548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 665/1310] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:11.638322 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c102f-3cef-449a-b2ad-c3294e3c4c79 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:11.640108 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-9a7c102f-3cef-449a-b2ad-c3294e3c4c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:11.887490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c102f-3cef-449a-b2ad-c3294e3c4c79 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:12.019249 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:12.019930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:12.020220 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:12.020671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:12.023022 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:12.093875 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:12.160012 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:12.161470 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbda6fa-0a23-4a7a-b8d0-016cbf921b48 service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:12.183659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 646/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:15.040274 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7920f6-d780-4d22-9959-1a0b2c6fd92e None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:15.040870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7920f6-d780-4d22-9959-1a0b2c6fd92e None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:15.040870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7920f6-d780-4d22-9959-1a0b2c6fd92e None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:15.046783 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 666/1312] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:15.048341 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1b158-2630-481e-b40b-b2a8f4d65ce0 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:15.048968 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1b158-2630-481e-b40b-b2a8f4d65ce0 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:15.048968 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1b158-2630-481e-b40b-b2a8f4d65ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:15.171734 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-f5e1b158-2630-481e-b40b-b2a8f4d65ce0 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:15.172904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-f5e1b158-2630-481e-b40b-b2a8f4d65ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:15.281612 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 647/1313] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:15.344781 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266df26-cae2-45cc-96aa-a8acad3d376a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:15.345285 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266df26-cae2-45cc-96aa-a8acad3d376a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:15.345733 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266df26-cae2-45cc-96aa-a8acad3d376a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:15.347613 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 667/1314] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 19 15:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:15.356789 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09e3a1-c3dd-4020-9f8d-7670c4dfd71c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:15.357186 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09e3a1-c3dd-4020-9f8d-7670c4dfd71c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:15.357408 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09e3a1-c3dd-4020-9f8d-7670c4dfd71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:15.369072 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:15.448146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-9c09e3a1-c3dd-4020-9f8d-7670c4dfd71c None None] MFA Rules not processed for user `abe824c999d34757b0c84e96bfa3d1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:15.449344 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-9c09e3a1-c3dd-4020-9f8d-7670c4dfd71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:15.540110 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 648/1315] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 19 15:07:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:15.550270 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266df26-cae2-45cc-96aa-a8acad3d376a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:15.551902 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-2266df26-cae2-45cc-96aa-a8acad3d376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:15.674879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266df26-cae2-45cc-96aa-a8acad3d376a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:15.757498 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:15.758564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:15.759358 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:15.759775 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:15.760162 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:15.762244 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:15.894378 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:15.896150 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3c7d-67c8-42be-b076-9e2b5d6fb5f0 service neutron] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:15.919291 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 668/1316] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:18.698711 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77505d-5bee-46b8-89d3-358dab8f1bee None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:18.699629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77505d-5bee-46b8-89d3-358dab8f1bee None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:18.699629 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77505d-5bee-46b8-89d3-358dab8f1bee None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:18.701874 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 649/1317] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:18.708420 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5b8f6-8238-499c-90be-754accdb21d0 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:18.708858 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5b8f6-8238-499c-90be-754accdb21d0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:18.708994 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5b8f6-8238-499c-90be-754accdb21d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:18.786813 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-0be5b8f6-8238-499c-90be-754accdb21d0 None None] MFA Rules not processed for user `96c2186a1d27432e8e831c999d91f365`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:18.788018 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-0be5b8f6-8238-499c-90be-754accdb21d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:18.843546 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:18.902794 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 669/1318] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:18.946744 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77505d-5bee-46b8-89d3-358dab8f1bee None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:18.949368 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-ca77505d-5bee-46b8-89d3-358dab8f1bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:19.065522 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77505d-5bee-46b8-89d3-358dab8f1bee None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:19.068382 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:19.141975 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:19.142899 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:19.143268 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:19.143634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:19.146459 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:19.277062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:19.278538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb43143-32e7-435b-8aa1-0fc0463f4026 service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:19.297482 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 650/1319] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:22.079782 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e006-e8b3-4054-8659-fc9ce28d28f4 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:22.080360 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e006-e8b3-4054-8659-fc9ce28d28f4 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:22.080360 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e006-e8b3-4054-8659-fc9ce28d28f4 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:22.081651 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 670/1320] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:22.088497 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72035a4c-b9b3-49f1-bcd5-d9b44f9b1e2b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:22.089283 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72035a4c-b9b3-49f1-bcd5-d9b44f9b1e2b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:22.089490 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72035a4c-b9b3-49f1-bcd5-d9b44f9b1e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:22.137521 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:22.166409 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-72035a4c-b9b3-49f1-bcd5-d9b44f9b1e2b None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:22.167627 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-72035a4c-b9b3-49f1-bcd5-d9b44f9b1e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:22.269792 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 651/1321] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:22.308183 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0e006-e8b3-4054-8659-fc9ce28d28f4 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:22.309806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-e2a0e006-e8b3-4054-8659-fc9ce28d28f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:22.424349 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0e006-e8b3-4054-8659-fc9ce28d28f4 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:22.513820 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:22.514727 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:22.515086 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:22.515806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:22.518220 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:22.527186 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:22.635719 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:22.637188 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55c6aa-14e2-45ed-859d-81a088c9bfed service neutron] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:22.656187 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 671/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:25.818104 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633ec2-29d7-4417-9b12-1dd1eace2d94 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:25.818724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633ec2-29d7-4417-9b12-1dd1eace2d94 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:25.818724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633ec2-29d7-4417-9b12-1dd1eace2d94 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:25.820079 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 652/1323] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:25.825790 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df3a7b-64db-4784-9ac4-a6224cb48055 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:25.826242 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df3a7b-64db-4784-9ac4-a6224cb48055 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:25.826374 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df3a7b-64db-4784-9ac4-a6224cb48055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:25.896321 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-f2df3a7b-64db-4784-9ac4-a6224cb48055 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:25.897647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-f2df3a7b-64db-4784-9ac4-a6224cb48055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:25.963057 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:26.004527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 672/1324] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:26.041211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633ec2-29d7-4417-9b12-1dd1eace2d94 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:26.042816 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-8c633ec2-29d7-4417-9b12-1dd1eace2d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:26.151248 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633ec2-29d7-4417-9b12-1dd1eace2d94 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:26.164111 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:26.230875 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:26.231537 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:26.231824 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:26.232099 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:26.233806 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:26.352144 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:26.353634 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff83b2c-6a78-4692-b43b-bf25a22072ab service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:26.372159 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 653/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:28.633107 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a98cf-d1db-4546-9140-a41085dc21dd None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:28.633767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a98cf-d1db-4546-9140-a41085dc21dd None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:28.633767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a98cf-d1db-4546-9140-a41085dc21dd None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:28.635145 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 673/1326] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:28.641014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671ee2b-a7ac-4b4b-94ad-7f7875e7541c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:28.641487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671ee2b-a7ac-4b4b-94ad-7f7875e7541c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:28.641659 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671ee2b-a7ac-4b4b-94ad-7f7875e7541c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:28.692311 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:28.711414 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-3671ee2b-a7ac-4b4b-94ad-7f7875e7541c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:28.712647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-3671ee2b-a7ac-4b4b-94ad-7f7875e7541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:28.813828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 654/1327] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:28.849548 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a98cf-d1db-4546-9140-a41085dc21dd None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:28.851102 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-d28a98cf-d1db-4546-9140-a41085dc21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:28.962948 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a98cf-d1db-4546-9140-a41085dc21dd None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:29.041103 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:29.041817 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:29.042099 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:29.042376 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:29.044060 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:29.061628 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:29.160219 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:29.161729 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfa02c-0de0-41a3-9416-7e401342ac9c service neutron] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:29.181421 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 674/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:31.463006 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459254a0-f306-4d9d-8d34-f046524ca684 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:31.463589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459254a0-f306-4d9d-8d34-f046524ca684 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:31.463589 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459254a0-f306-4d9d-8d34-f046524ca684 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:31.465014 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 655/1329] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:31.471527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06481884-32b2-41ea-bd8b-97cf3ff36add None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:31.472272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06481884-32b2-41ea-bd8b-97cf3ff36add None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:31.472444 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06481884-32b2-41ea-bd8b-97cf3ff36add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:31.544438 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-06481884-32b2-41ea-bd8b-97cf3ff36add None None] MFA Rules not processed for user `96c2186a1d27432e8e831c999d91f365`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:31.545647 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-06481884-32b2-41ea-bd8b-97cf3ff36add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:31.621622 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:31.656212 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 675/1330] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:31.692981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459254a0-f306-4d9d-8d34-f046524ca684 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:31.694582 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-459254a0-f306-4d9d-8d34-f046524ca684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:31.803173 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459254a0-f306-4d9d-8d34-f046524ca684 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:31.823664 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:31.883930 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:31.884567 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:31.884861 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:31.885146 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:31.886952 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:31.999362 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:32.000828 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1014d6-dba4-4ad0-8f92-50d63107f51e service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:32.019948 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 656/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:34.477287 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480bc2f-2d82-43e4-9457-623a1ff241f7 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:34.478175 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480bc2f-2d82-43e4-9457-623a1ff241f7 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:34.478381 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480bc2f-2d82-43e4-9457-623a1ff241f7 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:34.480461 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 676/1332] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:34.486036 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35730eb7-c7f9-465b-be38-c22e4e178d35 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:34.486474 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35730eb7-c7f9-465b-be38-c22e4e178d35 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:34.486576 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35730eb7-c7f9-465b-be38-c22e4e178d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:34.554709 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:34.558196 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-35730eb7-c7f9-465b-be38-c22e4e178d35 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:34.560547 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-35730eb7-c7f9-465b-be38-c22e4e178d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:34.669289 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 657/1333] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:34.705251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480bc2f-2d82-43e4-9457-623a1ff241f7 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:34.706870 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-c480bc2f-2d82-43e4-9457-623a1ff241f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:34.818295 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480bc2f-2d82-43e4-9457-623a1ff241f7 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:34.893539 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:34.894226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:34.894607 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:34.895030 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:34.897025 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:34.928765 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:35.015010 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:35.016826 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fed889-131b-49e0-89c2-32de52f45942 service neutron] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:35.037261 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 677/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:37.309022 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3fa5d-cb89-49a8-9a3f-aaffcc42d3d3 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:37.309614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3fa5d-cb89-49a8-9a3f-aaffcc42d3d3 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:37.309614 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3fa5d-cb89-49a8-9a3f-aaffcc42d3d3 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:37.311037 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 658/1335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:37.317480 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227bd58-8aa4-40eb-bbfd-77086815ea28 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:37.317954 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227bd58-8aa4-40eb-bbfd-77086815ea28 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:37.318068 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227bd58-8aa4-40eb-bbfd-77086815ea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:37.392055 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-1227bd58-8aa4-40eb-bbfd-77086815ea28 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:37.393348 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-1227bd58-8aa4-40eb-bbfd-77086815ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:37.472247 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:37.498250 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 678/1336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:37.542669 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3fa5d-cb89-49a8-9a3f-aaffcc42d3d3 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:37.545151 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-8df3fa5d-cb89-49a8-9a3f-aaffcc42d3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:37.666449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3fa5d-cb89-49a8-9a3f-aaffcc42d3d3 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:37.696089 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:37.745137 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:37.745811 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:37.746096 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:37.746364 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:37.748023 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:37.861873 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:37.863312 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b1daf-e3e8-47d6-8c03-f4a580454996 service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:37.881658 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 659/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:40.559090 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2347497-1d8d-40af-a7d3-32786159a0b0 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:40.560028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2347497-1d8d-40af-a7d3-32786159a0b0 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:40.560028 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2347497-1d8d-40af-a7d3-32786159a0b0 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:40.562251 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 679/1338] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:40.568724 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03533d-ba31-4fe3-b4ae-58e2f5937e98 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:40.569211 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03533d-ba31-4fe3-b4ae-58e2f5937e98 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:40.569360 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03533d-ba31-4fe3-b4ae-58e2f5937e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:40.640449 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-ad03533d-ba31-4fe3-b4ae-58e2f5937e98 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:40.641749 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-ad03533d-ba31-4fe3-b4ae-58e2f5937e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:40.651218 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:40.751116 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 660/1339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:40.801622 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2347497-1d8d-40af-a7d3-32786159a0b0 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:40.804048 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-f2347497-1d8d-40af-a7d3-32786159a0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:40.918845 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2347497-1d8d-40af-a7d3-32786159a0b0 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:41.000210 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:41.000876 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:41.001159 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:41.001477 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:41.003161 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:41.040855 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:41.122669 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:41.124128 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c8f4b-21d8-434e-b681-3e7eb271a300 service neutron] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:41.143961 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 680/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:45.495062 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62297-62be-4952-ac5f-7a9ec29838cc None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:45.495649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62297-62be-4952-ac5f-7a9ec29838cc None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:45.495649 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62297-62be-4952-ac5f-7a9ec29838cc None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:45.497140 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 661/1341] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:45.502405 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db3a0-05ff-40cb-89fb-24147c05bc91 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:45.502842 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db3a0-05ff-40cb-89fb-24147c05bc91 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:45.502946 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db3a0-05ff-40cb-89fb-24147c05bc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:45.576806 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-906db3a0-05ff-40cb-89fb-24147c05bc91 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:45.578096 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-906db3a0-05ff-40cb-89fb-24147c05bc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:45.686027 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=100 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:45.702564 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 681/1342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:45.742000 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e62297-62be-4952-ac5f-7a9ec29838cc None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:45.744420 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-41e62297-62be-4952-ac5f-7a9ec29838cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:45.848210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e62297-62be-4952-ac5f-7a9ec29838cc None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:45.852026 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37e901-02ed-43c4-b96d-38e1cec3817e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:45.852674 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37e901-02ed-43c4-b96d-38e1cec3817e service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:45.852956 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37e901-02ed-43c4-b96d-38e1cec3817e service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:45.853253 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37e901-02ed-43c4-b96d-38e1cec3817e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:45.862973 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37e901-02ed-43c4-b96d-38e1cec3817e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:45.864422 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37e901-02ed-43c4-b96d-38e1cec3817e service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:45.874201 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 662/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:48.701454 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2a574-f6fb-45b1-8cfa-403472e8dd4a None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:48.702029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2a574-f6fb-45b1-8cfa-403472e8dd4a None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:48.702029 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2a574-f6fb-45b1-8cfa-403472e8dd4a None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:48.703306 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 682/1344] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:48.708921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876178e-7342-4637-80d4-754cd1da9bcb None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:48.709501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876178e-7342-4637-80d4-754cd1da9bcb None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:48.709501 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876178e-7342-4637-80d4-754cd1da9bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:48.742439 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-a876178e-7342-4637-80d4-754cd1da9bcb None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:48.744131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-a876178e-7342-4637-80d4-754cd1da9bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:48.776581 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:07:48.830332 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 663/1345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:48.867513 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2a574-f6fb-45b1-8cfa-403472e8dd4a None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:48.879481 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2a574-f6fb-45b1-8cfa-403472e8dd4a None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:48.883777 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c123e6-d1a2-4c5d-8274-33534f1b2f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:48.884431 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c123e6-d1a2-4c5d-8274-33534f1b2f82 service neutron] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:48.884715 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c123e6-d1a2-4c5d-8274-33534f1b2f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:48.884985 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c123e6-d1a2-4c5d-8274-33534f1b2f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:48.895136 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c123e6-d1a2-4c5d-8274-33534f1b2f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:48.896624 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c123e6-d1a2-4c5d-8274-33534f1b2f82 service neutron] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:48.907527 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 683/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:53.160921 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a673201-fdfd-46ea-b314-417e237cdb0e None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:53.161559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a673201-fdfd-46ea-b314-417e237cdb0e None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:53.161559 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a673201-fdfd-46ea-b314-417e237cdb0e None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:53.162946 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 664/1347] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:53.168444 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa160ab1-d554-4934-b611-1b1bb631d8cd None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:53.168898 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa160ab1-d554-4934-b611-1b1bb631d8cd None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:53.169026 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa160ab1-d554-4934-b611-1b1bb631d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:53.187542 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-aa160ab1-d554-4934-b611-1b1bb631d8cd None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:53.188811 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-aa160ab1-d554-4934-b611-1b1bb631d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:53.197286 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 684/1348] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:07:53.234950 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a673201-fdfd-46ea-b314-417e237cdb0e None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:07:53.247487 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a673201-fdfd-46ea-b314-417e237cdb0e None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:07:53.251777 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e18e-a377-479c-90ac-1c1d4fa441c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe824c999d34757b0c84e96bfa3d1a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:07:53.252358 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e18e-a377-479c-90ac-1c1d4fa441c9 service neutron] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:53.252639 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e18e-a377-479c-90ac-1c1d4fa441c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:53.252920 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e18e-a377-479c-90ac-1c1d4fa441c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:53.262460 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e18e-a377-479c-90ac-1c1d4fa441c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:07:53.263897 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e18e-a377-479c-90ac-1c1d4fa441c9 service neutron] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:07:53.278109 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 665/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:07:58.771740 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30556fb-76d7-45f9-8d7a-5c5d55cbf65f None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:58.772736 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30556fb-76d7-45f9-8d7a-5c5d55cbf65f None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:58.773362 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30556fb-76d7-45f9-8d7a-5c5d55cbf65f None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:58.775272 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 685/1350] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 15:07:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:07:58.782389 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aed657-b6f7-4fc3-bb84-f74cc55e2a6b None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:07:58.783488 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aed657-b6f7-4fc3-bb84-f74cc55e2a6b None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:07:58.784086 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aed657-b6f7-4fc3-bb84-f74cc55e2a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:07:58.869556 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-47aed657-b6f7-4fc3-bb84-f74cc55e2a6b None None] MFA Rules not processed for user `624b80e56c8c44e8bc4c1b43089d50f6`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:07:58.871866 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-47aed657-b6f7-4fc3-bb84-f74cc55e2a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:07:58.969768 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 666/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 15:07:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:03.182156 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=13 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:08:03.390464 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55979-4ece-4e17-b294-aec3f0c5ef65 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:03.390927 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55979-4ece-4e17-b294-aec3f0c5ef65 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:03.391352 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55979-4ece-4e17-b294-aec3f0c5ef65 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:03.393215 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 686/1352] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 15:08:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:03.398766 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db452df-41fc-4239-8b90-de5ff1d4136a None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:03.399291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db452df-41fc-4239-8b90-de5ff1d4136a None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:03.399291 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db452df-41fc-4239-8b90-de5ff1d4136a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:03.418269 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-9db452df-41fc-4239-8b90-de5ff1d4136a None None] MFA Rules not processed for user `624b80e56c8c44e8bc4c1b43089d50f6`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:03.419517 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-9db452df-41fc-4239-8b90-de5ff1d4136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:03.428622 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 667/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 15:08:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:04.981732 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f7fa3-5e3b-43f0-96b2-10a3da9f0240 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:04.982366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f7fa3-5e3b-43f0-96b2-10a3da9f0240 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:04.982366 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f7fa3-5e3b-43f0-96b2-10a3da9f0240 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:04.983940 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 687/1354] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 19 15:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:04.989125 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3ed2-a5e4-4d57-97ff-be4f4e4381f9 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:04.989640 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3ed2-a5e4-4d57-97ff-be4f4e4381f9 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:04.989842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3ed2-a5e4-4d57-97ff-be4f4e4381f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:05.009448 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-968a3ed2-a5e4-4d57-97ff-be4f4e4381f9 None None] MFA Rules not processed for user `624b80e56c8c44e8bc4c1b43089d50f6`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:05.010675 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-968a3ed2-a5e4-4d57-97ff-be4f4e4381f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:05.021051 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 668/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 19 15:08:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:08.940809 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46473a14-2199-46cd-8688-9eba5b324d23 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:08.941453 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46473a14-2199-46cd-8688-9eba5b324d23 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:08.941453 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46473a14-2199-46cd-8688-9eba5b324d23 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:08.942834 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 688/1356] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:08.948337 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f2ae1-db98-4945-b5d9-e5f26979e05c None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:08.948753 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f2ae1-db98-4945-b5d9-e5f26979e05c None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:08.948851 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f2ae1-db98-4945-b5d9-e5f26979e05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:08.968063 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-e50f2ae1-db98-4945-b5d9-e5f26979e05c None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:08.969313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-e50f2ae1-db98-4945-b5d9-e5f26979e05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:08.977597 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 669/1357] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:09.047918 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fdd0b-3a5a-44f3-983e-ae524bd875e8 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:09.048277 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fdd0b-3a5a-44f3-983e-ae524bd875e8 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:09.048503 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fdd0b-3a5a-44f3-983e-ae524bd875e8 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:09.051144 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 689/1358] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 15:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:09.060162 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6877a0-873c-4750-b53d-a66491919ba5 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:09.060904 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6877a0-873c-4750-b53d-a66491919ba5 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:09.061686 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6877a0-873c-4750-b53d-a66491919ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:09.108662 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=100 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:08:09.117206 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-9a6877a0-873c-4750-b53d-a66491919ba5 None None] MFA Rules not processed for user `b764ac4c637749019a830d8cf3a22e09`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:09.118607 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-9a6877a0-873c-4750-b53d-a66491919ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:09.208728 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 670/1359] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 15:08:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:09.216744 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fdd0b-3a5a-44f3-983e-ae524bd875e8 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:08:09.232049 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fdd0b-3a5a-44f3-983e-ae524bd875e8 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:08:09.238037 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c83309-554a-469f-b827-4f200033d51d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b764ac4c637749019a830d8cf3a22e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:08:09.238958 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83309-554a-469f-b827-4f200033d51d service nova] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:09.239424 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83309-554a-469f-b827-4f200033d51d service nova] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:09.239913 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83309-554a-469f-b827-4f200033d51d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:09.248779 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c83309-554a-469f-b827-4f200033d51d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:08:09.250760 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c83309-554a-469f-b827-4f200033d51d service nova] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:08:09.261593 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 690/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:08:11.684550 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b953f-226b-4cad-b3a9-f237b6d20262 None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:11.685095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b953f-226b-4cad-b3a9-f237b6d20262 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:11.685095 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b953f-226b-4cad-b3a9-f237b6d20262 None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:11.686538 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 671/1361] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:11.692203 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939274dc-167c-4f96-9dd3-96db6165eb07 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:11.692646 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939274dc-167c-4f96-9dd3-96db6165eb07 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:11.692768 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939274dc-167c-4f96-9dd3-96db6165eb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:11.712135 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-939274dc-167c-4f96-9dd3-96db6165eb07 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:11.713471 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-939274dc-167c-4f96-9dd3-96db6165eb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:11.722165 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 691/1362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:11.767335 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b953f-226b-4cad-b3a9-f237b6d20262 None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:08:11.782918 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b953f-226b-4cad-b3a9-f237b6d20262 None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:08:11.788951 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb2de5-14fa-4a1b-ae48-074d50c4fe52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b764ac4c637749019a830d8cf3a22e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:08:11.789647 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb2de5-14fa-4a1b-ae48-074d50c4fe52 service nova] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:11.789936 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb2de5-14fa-4a1b-ae48-074d50c4fe52 service nova] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:11.790213 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb2de5-14fa-4a1b-ae48-074d50c4fe52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:11.799210 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb2de5-14fa-4a1b-ae48-074d50c4fe52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:08:11.800826 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb2de5-14fa-4a1b-ae48-074d50c4fe52 service nova] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:08:11.811274 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 672/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:08:14.344172 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b9a5-b447-4e70-81ca-db43a3e04b98 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:14.344767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b9a5-b447-4e70-81ca-db43a3e04b98 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:14.344767 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b9a5-b447-4e70-81ca-db43a3e04b98 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:14.346456 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 692/1364] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:14.351981 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b299b-ea41-4732-bd85-5b82bd931f4d None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:14.352447 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b299b-ea41-4732-bd85-5b82bd931f4d None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:14.352759 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b299b-ea41-4732-bd85-5b82bd931f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:14.371996 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-c37b299b-ea41-4732-bd85-5b82bd931f4d None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:14.373224 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-c37b299b-ea41-4732-bd85-5b82bd931f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:14.382631 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 673/1365] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:14.428862 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad437064-6cf9-4c71-8011-c5e0e7d8d707 None None] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:14.429110 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad437064-6cf9-4c71-8011-c5e0e7d8d707 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:14.429462 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad437064-6cf9-4c71-8011-c5e0e7d8d707 None None] PATH_INFO: `/` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:14.430827 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 693/1366] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 19 15:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:14.436758 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a804faa-5e7e-44c8-9b89-9fb5834a7da6 None None] REQUEST_METHOD: `POST` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:14.437157 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a804faa-5e7e-44c8-9b89-9fb5834a7da6 None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:14.437603 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a804faa-5e7e-44c8-9b89-9fb5834a7da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:14.469671 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.auth.core [None req-5a804faa-5e7e-44c8-9b89-9fb5834a7da6 None None] MFA Rules not processed for user `b764ac4c637749019a830d8cf3a22e09`. Rule list: `[]` (Enabled: `True`). {{(pid=277130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:14.471081 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.fernet_utils [None req-5a804faa-5e7e-44c8-9b89-9fb5834a7da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:14.489774 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 674/1367] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 19 15:08:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:14.499521 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad437064-6cf9-4c71-8011-c5e0e7d8d707 None None] Authenticating user token {{(pid=277131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:08:14.517182 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad437064-6cf9-4c71-8011-c5e0e7d8d707 None None] Validating token access rules against request {{(pid=277131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:08:14.523217 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153edf1-8e72-44f4-9f48-6fb212b47c59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b764ac4c637749019a830d8cf3a22e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:08:14.523879 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153edf1-8e72-44f4-9f48-6fb212b47c59 service nova] REQUEST_METHOD: `GET` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:14.524226 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153edf1-8e72-44f4-9f48-6fb212b47c59 service nova] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:14.524452 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153edf1-8e72-44f4-9f48-6fb212b47c59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:14.533685 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153edf1-8e72-44f4-9f48-6fb212b47c59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:08:14.535298 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153edf1-8e72-44f4-9f48-6fb212b47c59 service nova] RBAC: Authorization granted {{(pid=277131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:08:14.545758 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 694/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:08:15.420842 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870bc16-df24-4667-8832-0673905bf0ce None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:15.421305 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870bc16-df24-4667-8832-0673905bf0ce None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:15.421518 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870bc16-df24-4667-8832-0673905bf0ce None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:15.424736 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 675/1369] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:08:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:15.429922 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3a0ba-9906-428a-bfc2-db03d5643add None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:15.430478 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3a0ba-9906-428a-bfc2-db03d5643add None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:15.430574 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3a0ba-9906-428a-bfc2-db03d5643add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:15.450929 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-84e3a0ba-9906-428a-bfc2-db03d5643add None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:15.452296 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-84e3a0ba-9906-428a-bfc2-db03d5643add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:15.461336 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 695/1370] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:16.915143 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41fa0c-600e-4255-81ce-495aa211a8ae None None] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:16.915757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41fa0c-600e-4255-81ce-495aa211a8ae None None] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:16.915757 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41fa0c-600e-4255-81ce-495aa211a8ae None None] PATH_INFO: `/` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:16.917131 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 676/1371] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 19 15:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 15:08:16.922578 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3372a6d-4055-4072-bb77-94e413afcc53 None None] REQUEST_METHOD: `POST` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:16.923041 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3372a6d-4055-4072-bb77-94e413afcc53 None None] SCRIPT_NAME: `/identity` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:16.923157 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3372a6d-4055-4072-bb77-94e413afcc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:16.943852 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.auth.core [None req-d3372a6d-4055-4072-bb77-94e413afcc53 None None] MFA Rules not processed for user `c7ff97c0da314994a8835ccf5732ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=277131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 15:08:16.945058 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG keystone.common.fernet_utils [None req-d3372a6d-4055-4072-bb77-94e413afcc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 15:08:16.954095 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: [pid: 277131|app: 0|req: 696/1372] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 19 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 15:08:16.997279 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41fa0c-600e-4255-81ce-495aa211a8ae None None] Authenticating user token {{(pid=277130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 15:08:17.012999 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41fa0c-600e-4255-81ce-495aa211a8ae None None] Validating token access rules against request {{(pid=277130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 15:08:17.019313 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43347bb-bc74-4fc3-b3b9-6c3f8179d328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b764ac4c637749019a830d8cf3a22e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '369ac6c738f848fa8c426fb690d2db7f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 15:08:17.019958 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43347bb-bc74-4fc3-b3b9-6c3f8179d328 service nova] REQUEST_METHOD: `GET` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 15:08:17.020246 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43347bb-bc74-4fc3-b3b9-6c3f8179d328 service nova] SCRIPT_NAME: `/identity` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 15:08:17.020527 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43347bb-bc74-4fc3-b3b9-6c3f8179d328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 15:08:17.029005 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43347bb-bc74-4fc3-b3b9-6c3f8179d328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 15:08:17.030644 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43347bb-bc74-4fc3-b3b9-6c3f8179d328 service nova] RBAC: Authorization granted {{(pid=277130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 15:08:17.040310 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: [pid: 277130|app: 0|req: 677/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 19 15:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 15:08:26.938547 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: DEBUG dbcounter [-] [277131] Writing DB stats keystone:SELECT=21 {{(pid=277131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:08:27.036579 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: DEBUG dbcounter [-] [277130] Writing DB stats keystone:SELECT=54 {{(pid=277130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 15:08:43.252215 dv-cinder-842542-1-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 15:08:43.252863 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: Thu May 19 15:08:43 2022 - graceful shutdown triggered... May 19 15:08:43.253570 dv-cinder-842542-1-iscsi devstack@keystone.service[277131]: Gracefully killing worker 2 (pid: 277131)... May 19 15:08:43.254306 dv-cinder-842542-1-iscsi devstack@keystone.service[277130]: Gracefully killing worker 1 (pid: 277130)... May 19 15:08:44.254101 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: worker 1 buried after 1 seconds May 19 15:08:44.254101 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: worker 2 buried after 1 seconds May 19 15:08:44.254101 dv-cinder-842542-1-iscsi devstack@keystone.service[277118]: goodbye to uWSGI. May 19 15:08:44.256783 dv-cinder-842542-1-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 15:08:44.257748 dv-cinder-842542-1-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.